- Ease of programming (development and deployment)
- Running Speed
- Truly open (open source with good governance)
- Popularity

University of Michigan - Ann Arbor

Deep learning frameworks provide specialized software tools that streamline the construction, training, and deployment of neural networks. The availability of these efficient frameworks has made the design and implementation of whole system optimization significantly easier, which is a critical component in achieving high performance for complex deep learning pipelines.

Deep Learning Frameworks

Criterion for Choosing a Deep Learning Framework

Deep Learning Frameworks References

Keras is an API designed for human beings, not machines. Keras follows best practices for reducing cognitive load: it offers consistent & simple APIs, it minimizes the number of user actions required for common use cases, and it provides clear & actionable error messages. It also has extensive documentation and developer guides.

https://keras.io/

Keras

TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications.

https://www.tensorflow.org/

TensorFlow

PyTorch is an open source machine learning framework that accelerates the path from research prototyping to production deployment.

https://pytorch.org/

PyTorch

Theano is a Python library that allows you to define, optimize, and efficiently evaluate mathematical expressions involving multi-dimensional arrays. It is built on top of NumPy.

https://pypi.org/project/Theano/

Theano

PaddlePaddle, as the only independent R&D deep learning platform in China, has been officially open-sourced to professional communities since 2016. It is an industrial platform with advanced technologies and rich features that cover core deep learning frameworks, basic model libraries, end-to-end development kits, tools & components as well as service platforms. PaddlePaddle is originated from industrial practices with dedication and commitments to industrialization. It has been widely adopted by a wide range of sectors including manufacturing, agriculture, enterprise service, and so on while serving more than 2.3 million developers. With such advantages, PaddlePaddle has helped an increasing number of partners commercialize AI.

https://pypi.org/project/paddlepaddle/

PaddlePaddle

A truly open source deep learning framework suited for flexible research prototyping and production.

https://mxnet.apache.org/


mxnet

Lasagne is a lightweight library to build and train neural networks in Theano.

https://lasagne.readthedocs.io/

Lasagne

Eclipse Deeplearning4j is the first commercial-grade, open-source, distributed deep-learning library written for Java and Scala. Integrated with Hadoop and Apache Spark, DL4J brings AI to business environments for use on distributed GPUs and CPUs.

https://deeplearning4j.org/

DL4J

The Microsoft Cognitive Toolkit (CNTK) is an open-source toolkit for commercial-grade distributed deep learning. It describes neural networks as a series of computational steps via a directed graph. CNTK allows the user to easily realize and combine popular model types such as feed-forward DNNs, convolutional neural networks (CNNs) and recurrent neural networks (RNNs/LSTMs). CNTK implements stochastic gradient descent (SGD, error backpropagation) learning with automatic differentiation and parallelization across multiple GPUs and servers.

https://docs.microsoft.com/en-us/cognitive-toolkit/

CNTK

Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by Berkeley AI Research (BAIR) and by community contributors. Yangqing Jia created the project during his PhD at UC Berkeley. Caffe is released under the BSD 2-Clause license.

https://caffe.berkeleyvision.org/

Caffe

1. Style

If you’re a Python programmer, then PyTorch will feel easy to pick up. It works the way you’d expect it to, right out of the box.

On the other hand, more coding languages are supported in TensorFlow than in PyTorch, which has a C++ API. You can use TensorFlow in both JavaScript and Swift. If you don’t want to write much low-level code, then Keras abstracts away a lot of the details for common use cases so you can build TensorFlow models without sweating the details.

2. Data and Model

What models are you using? If you want to use a specific pretrained model, like BERT or DeepDream, then you should research what it’s compatible with. Some pretrained models are available in only one library or the other, and some are available on both. The Model Garden and the PyTorch and TensorFlow hubs are also good resources to check.

What data do you need? If you want to use preprocessed data, then it may already be built into one library or the other. Check the docs to see—it will make your development go faster!

3. Project Goal

Where will your model live? If you want to deploy a model on mobile devices, then TensorFlow is a good bet because of TensorFlow Lite and its Swift API. For serving models, TensorFlow has tight integration with Google Cloud, but PyTorch is integrated into TorchServe on AWS. If you want to enter Kaggle competitions, then Keras will let you quickly iterate over experiments.

Think about these questions and examples at the outset of your project. Nail down the two or three most important components, and either TensorFlow or PyTorch will emerge as the right choice.

https://realpython.com/pytorch-vs-tensorflow/

PyTorch vs TensorFlow

High-level APIs in deep learning frameworks offer significant convenience by abstracting complex and potentially unstable operations, such as numerical stability, and allowing users to implement models concisely. This abstraction dramatically increases accessibility, enabling individuals without advanced statistical training to utilize deep learning. However, relying solely on high-level APIs presents drawbacks: it discourages the creation of custom, non-standard components and leaves practitioners unequipped to debug edge cases when the framework's abstractions fail. To invent novel architectures that frameworks do not natively support, deep learning practitioners must understand both the fundamental, from-scratch implementations and the concise, framework-based versions.

Learn Before

Related