PyTorch vs TensorFlow: A Small Experiment

We started with four practical questions: Is it worth using TensorFlow? Would PyTorch be sufficient? What are the real advantages and disadvantages? And how much do these differences matter in a typical project? To answer these questions, however, we need to conduct a small experiment.

Experiment Setup

The experiment was designed as a comparison of the two frameworks on the same image recognition task. The CIFAR-10 dataset was used, containing 60,000 images of size 32 × 32 pixels across ten classes. The architectures tested were ResNet50, VGG19, and MobileNet. Training was conducted for 5 epochs with a batch size of 32 on Intel i7-11370 and NVIDIA GeForce RTX 3060 hardware.

Implementation Procedure

The experiment followed the same logic in both frameworks:

1. Loading data (train/test) from the CIFAR-10 dataset
2. Initializing the model (ResNet50/VGG19/MobileNet)
3. Pre-training predictions – used as a baseline
4. Model fine-tuning on the dataset
5. Predictions after training

What was monitored during the test

Accuracy after training was only one part of the picture. The purpose of the test was also to observe how the frameworks behave during computation. The following parameters were monitored:

• Model accuracy before and after fine-tuning
• Total training time
• GPU utilization during runtime and framework behavior between epochs
• GPU memory handling
• Developer experience: installation, code readability, and documentation

Experiment results

Model accuracy before and after fine-tuning

Total training time

GPU utilization during runtime and framework behavior between epochs

These graphs show that TensorFlow has significantly higher overhead around epochs. This is often due to processes that occur at the end of each epoch, such as validation, weight updates, and callbacks.

In shorter runs, these overheads are more noticeable. In longer training sessions (tens to hundreds of epochs), the fixed overhead tends to diminish, and the difference becomes less significant.

GPU memory handling

A significant difference in the strategies of the two frameworks can be observed here. PyTorch is more predictable during debugging, while TensorFlow tends to allocate memory in blocks.

Developer experience: installation, code readability, and documentation