R interface to TensorFlow Hub

We are pleased to announce that the first version of tfhub is now on CRAN. tfhub is an R interface to TensorFlow Hub – a library for the publication, discovery, and consumption of reusable parts of machine learning models. A module is a self-contained piece of a TensorFlow graph, along with its weights and assets, that can be reused across different tasks in a process known as transfer learning.

The CRAN version of tfhub can be installed with:

After installing the R package you need to install the TensorFlow Hub python package. You can do it by running:

Getting started

The essential function of tfhub is layer_hub which works just like a keras layer but allows you to load a complete pre-trained deep learning model.

For example you can:

library(tfhub)
layer_mobilenet <- layer_hub(
  handle = "https://tfhub.dev/google/tf2-preview/mobilenet_v2/classification/4"
)

This will download the MobileNet model pre-trained on the ImageNet dataset. tfhub models are cached locally and don’t need to be downloaded the next time you use the same model.

You can now use layer_mobilenet as a usual Keras layer. For example you can define a model:

library(keras)
input <- layer_input(shape = c(224, 224, 3))
output <- layer_mobilenet(input)
model <- keras_model(input, output)
summary(model)

Model: "model"
____________________________________________________________________
Layer (type)                  Output Shape               Param #    
====================================================================
input_2 (InputLayer)          [(None, 224, 224, 3)]      0          
____________________________________________________________________
keras_layer_1 (KerasLayer)    (None, 1001)               3540265    
====================================================================
Total params: 3,540,265
Trainable params: 0
Non-trainable params: 3,540,265
____________________________________________________________________

This model can now be used to predict Imagenet labels for an image. For example, let’s see the results for the famous Grace Hopper’s photo:

img <- image_load("https://blogs.rstudio.com/tensorflow/posts/images/grace-hopper.jpg", target_size = c(224,224)) %>% 
  image_to_array()
img <- img/255
dim(img) <- c(1, dim(img))
pred <- predict(model, img)
imagenet_decode_predictions(pred[,-1,drop=FALSE])[[1]]

  class_name class_description    score
1  n03763968  military_uniform 9.760404
2  n02817516          bearskin 5.922512
3  n04350905              suit 5.729345
4  n03787032       mortarboard 5.400651
5  n03929855       pickelhaube 5.008665

TensorFlow Hub also offers many other pre-trained image, text and video models.
All possible models can be found on the TensorFlow hub website.

You can find more examples of layer_hub usage in the following articles on the TensorFlow for R website:

Usage with Recipes and the Feature Spec API

tfhub also offers recipes steps to make
it easier to use pre-trained deep learning models in your machine learning workflow.

For example, you can define a recipe that uses a pre-trained text embedding model with:

rec <- recipe(obscene ~ comment_text, data = train) %>%
  step_pretrained_text_embedding(
    comment_text,
    handle = "https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim-with-oov/1"
  ) %>%
  step_bin2factor(obscene)

You can see a complete running example here.

You can also use tfhub with the new Feature Spec API implemented in tfdatasets. You can see a complete example here.

We hope our readers have fun experimenting with Hub models and/or can put them to good use. If you run into any problems, let us know by creating an issue in the tfhub repository

Reuse

Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don’t fall under this license and can be recognized by a note in their caption: “Figure from …”.

Citation

For attribution, please cite this work as

Falbel (2019, Dec. 18). Posit AI Blog: tfhub: R interface to TensorFlow Hub. Retrieved from https://blogs.rstudio.com/tensorflow/posts/2019-12-18-tfhub-0.7.0/

BibTeX citation

@misc{tfhub,
  author = {Falbel, Daniel},
  title = {Posit AI Blog: tfhub: R interface to TensorFlow Hub},
  url = {https://blogs.rstudio.com/tensorflow/posts/2019-12-18-tfhub-0.7.0/},
  year = {2019}
}