4.2 Metalhead.jl

Let us start out by installing Metalhead in the package manager by doing add Metalhead. Then we can type using Flux, MetalHead into a Julia terminal session. Metalhead provides a number of different model types like resnet, vgg, and more. Over the course of your deep learning experience, you will become more familiar with these model types as they represent some of the most common models for transfer learning. Before we dive into actually using pre-trained models, we will first take a quick look at the model structure which we get from Metalhead and Flux.

julia> model = ResNet50(pretrain=false)
ResNet(
  Chain(
    Chain(
      Conv((7, 7), 3 => 64, pad=3, stride=2),  # 9_472 parameters
      BatchNorm(64, relu),              # 128 parameters, plus 128
      MaxPool((3, 3), pad=1, stride=2),
      Parallel(
        Metalhead.var"#18#20"(),
        Chain(
          Conv((1, 1), 64 => 64, bias=false),  # 4_096 parameters
          BatchNorm(64, relu),          # 128 parameters, plus 128
          Conv((3, 3), 64 => 64, pad=1, bias=false),  # 36_864 parameters
          BatchNorm(64, relu),          # 128 parameters, plus 128
          Conv((1, 1), 64 => 256, bias=false),  # 16_384 parameters
          BatchNorm(256),               # 512 parameters, plus 512
        ),
        Chain(
          Conv((1, 1), 64 => 256, bias=false),  # 16_384 parameters
          BatchNorm(256),               # 512 parameters, plus 512
        ),
      ),
      ...
      ...
      ...
    Chain(
      AdaptiveMeanPool((1, 1)),
      Flux.flatten,
      Dense(2048, 1000),                # 2_049_000 parameters
    ),
  ),
)         # Total: 162 trainable arrays, 25_557_096 parameters,
          # plus 106 non-trainable, 53_120 parameters, summarysize 97.749 MiB.

Now that is a lot to take in (even though most of the model has been omitted for space reasons), but at a high level, it represents the structure of the ResNet50 model as defined in Flux. Notably, the 50 in ResNet50 comes from the fact that the model has 50 layers. There are other ResNet like models with different numbers of layers but with the same overall structure.



CC BY-NC-SA 4.0 Logan Kilpatrick