Using PyTorch on iOS: A Comprehensive Guide
Welcome to our tutorial on using PyTorch on iOS. I'm Tao, a software engineer on the PyTorch Mobile team, and I'll be guiding you through this comprehensive guide.
Improvements and New Features in PyTorch 1.9
Before we dive into the demo, let's quickly introduce some of the new features we released in PyTorch 1.9. These include mobile interpreter, Metal back-end, custom builds, and Torch Vision Ops. We'll be covering most of these features in this demo. If you want to learn more, please refer to our in-depth tutorials and demo apps.
Setting Up the Project
To follow along, you need PyTorch Ice Cream version 12 or higher on macOS, Cocoapods installed on your machine, and an Apple Developer account with an iOS device running at least iOS 12. You should also be using PyTorch 1.9 or higher. Let's set up the project to include Python iOS Runtime.
First, we need to install PyTorch Ice Cream. Once installed, open a terminal and run `pip3 install torch torchvision` to install the required packages. Next, create a new folder for your project and navigate to it in the terminal. Create a new file called `requirements.txt` and add `torch torchvision numpy pandas` to it.
Now, let's install Cocoapods by running `gem install cocoapods` in the terminal. After installing Cocoapods, we can install PyTorch iOS Runtime using `pod init`. This will create a new `Podfile` in your project directory.
We need to specify that we want to use PyTorch 1.9 and enable the Python iOS Runtime by adding the following lines to our `Podfile`:
```ruby
target 'YourProject' do
pod 'PyTorch/Python', :path => '../Python'
end
```
Now, let's run `pod install` to install PyTorch iOS Runtime.
Resizing and Normalizing Images
To prepare the input image for our model, we need to resize it to 2x2 by 2x2 and normalize it. We can do this using the following Python code:
```python
import torch
from PIL import Image
def resize_image(image):
return image.resize((2, 2))
def normalize_image(image):
return image.convert('L').point(lambda x: min(255, max(0, x)))
image = Image.open('input_image.jpg')
resized_image = resize_image(image)
normalized_image = normalize_image(resized_image)
# Convert the normalized image to a tensor
tensor = torch.from_numpy(normalized_image).float()
```
Creating and Training the Model
Next, we'll create and train our model using PyTorch. We'll use a simple convolutional neural network (CNN) as an example.
```python
import torch.nn as nn
class CNN(nn.Module):
def __init__(self):
super(CNN, self).__init__()
self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
self.pool = nn.MaxPool2d(2, 2)
self.fc1 = nn.Linear(320, 50)
self.fc2 = nn.Linear(50, 10)
def forward(self, x):
x = self.pool(torch.relu(self.conv1(x)))
x = x.view(-1, 320)
x = torch.relu(self.fc1(x))
x = self.fc2(x)
return x
model = CNN()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
for epoch in range(10):
optimizer.zero_grad()
outputs = model(tensor)
loss = criterion(outputs, torch.tensor([1]))
loss.backward()
optimizer.step()
print('Epoch {}: Loss = {:.4f}'.format(epoch+1, loss.item()))
```
Running the Model on GPU
To run our model on the GPU, we need to install PyTorch Nightly Builds. We'll also update our Cocoapods to use the nightly build.
```bash
pip3 install torch torchvision --upgrade --pre
pod update --repo-url https://github.com/pytorch/pytorch.git --update-binary-dependencies
```
Next, let's make some changes to our Python script to run the model on the GPU. We'll add a `metal` parameter to the `optimize_for_mobile` function and specify the GPU back-end as `Metal`.
```python
import torch
def optimize_for_mobile(model, tensor):
return model.optimize_for_mobile(metal=True)
model = CNN()
tensor = torch.randn(1, 1, 28, 28)
gpu_model = optimize_for_mobile(model, tensor)
# Update Cocoapods to use the nightly build
pod update --repo-url https://github.com/pytorch/pytorch.git --update-binary-dependencies
# Open the Xcode project and add the GPU back-end
```
Using Custom Build to Reduce Binary Size
To reduce the binary size of our app, we can create a custom library that only contains the operators we need. We'll use PyTorch's `operator` module to dump the root operators from our model and save them to a YAML file.
```python
import torch
from pytorchOperators import get_root_operators
def dump_operators(model):
operators = get_root_operators(model)
with open('operators.yaml', 'w') as f:
for operator in operators:
f.write(str(operator) + '\n')
model = CNN()
dump_operators(model)
# Use the YAML file to build our custom library
```
We'll use PyTorch's `operator` module to generate a Python repository checkout from GitHub. We'll then run the `build` command on the screen.
```bash
python3 -m pip install --pre install-pytorch-operators
git clone https://github.com/pytorch/pytorch.git pytorch-operators
cd pytorch-operators
./configure --with-oplayers=/path/to/operations.yaml --prefix=/path/to/build/dir
```
We can then add the custom library to our `Podfile` using the following lines:
```ruby
target 'YourProject' do
pod 'PyTorch/Python', :path => '../Python'
end
target 'YourProject' do
pod 'PyTorch/Operators', :path => '../operators'
end
```
Now, let's run `pod install` to install our custom library.
Conclusion
In this tutorial, we've learned how to create and train a PyTorch model using a convolutional neural network (CNN) as an example. We've also learned how to run the model on the GPU and use a custom library to reduce the binary size of our app.