Content
1. Python Landscape for Machine Learning and Deep Learning 2. What's TensorFlow? 3. What is Keras? 4. History of Keras and TensorFlow 5. Deep-Learning-Workspace 6. Introduction to Tensor <- Go BackUniversity of Michigan at Ann Arbor
Last Edit Date: 02/21/2023
Disclaimer and Term of Use:
We do not guarantee the accuracy and completeness of the summary content. Some of the course material may not be included, and some of the content in the summary may not be correct. You should use this file properly and legally. We are not responsible for any results from using this file
This personal note is adapted from Deep Learning with Python and Professor Ambuj Tewari. Please contact us to delete this file if you think your rights have been violated.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Python is an interpreted high-level general-purpose programming language.
NumPy is a Python library for manipulation multi-dimensional arrays and matrices. Allows Python to be used for numerical computing.
Matplotlib is a Python library for creating plots. Works nicely with Numpy. Its pyplot module provides a MATLAB-like interface.
Scikit-learn (SK-learn) is a Python library built on top of NumPy and Matplotlib. This is the main library for machine learning especially for methods not based on neural networks.
TensorFlow is a Python library developed by Google for training deep learning models on CPUs, GPUs, and TPUs. TF2 was released in 2019.
Keras is built on top of TensorFlow and provides consistent & simple APIs, minimuzes the number of user actions required for common use cases, and provides clear & actionable error messages. The author of Keras, Francois Chollet, is also the author of the book Deep Learning with Python.
TensorFlow allows engineers/researchers to manipulate numerica tensors. But TensorFlow goes far beyond the scope of NumPy:
Gradient: It can automatically computer the gradient of any differentiable expression, making it highly suitable for machine learning.
Hardware Acceleration: It can run only only on CPUs, but also GPUs and TPUs.
Compatibility: Computation defined in TensorFlow can be easily distributed across many machines.
Easy Deployment: TensorFlow programs can be exported to other runtimes, such as C++, JavaScript, or TensorFlow Lite, etc.
TensorFlow is an ecosystem, it has:
TF-Agents for reinforcement learning research
TensorFlow Hub for pretrained models
TensorFlow Quantum for quantum machine learning
TensorFlow is a low-level tensor computing platform, and Keras is a high-level deep learning API.
Keras provides deep learning development such as layers, models, optimizers, losses, metrics, and so on.
TensorFlow provides manipulation infrastructure such as tensors, variables, automatic differentiation, distribution, and so on.
Note: CNTK (Microsoft) and MXNet (Amazon) were added as backends in 2017. Nowadays, both Theano and CNTK are out of development, and MXNet is not widely used outside of Amazon. Keras is back to being a single-backend API—on top of TensorFlow.
i. Hardware
A modern NVIDIA GPU is recommended, but not strictly necessary.
Some applications will be excruciatingly slow on CPU.
Even for applications that can be run on CPU, using GPU will increase the speed 5 to 10 times.
There are three options to get and use GPU:
By and install a physical NVIDIA GPU on your workstation.
Use GPU instances on Google Cluod or AWS EC2.
Use the free GPU runtime from Colaboratory (Colab), a hosted notebook service offered by Google.
ii. Jupyter notebook
It is the preferred way to run deep-learning experiments.
The name “Jupyter” is a strong reference to Galileo, who detailed his discovery of the Moons of Jupiter in his astronomical notebooks. The name is also a play on the languages Julia, Python, and R, which are pillars of the modern scientific world.
$${\huge Ju(lia)~~Py(thon)~~te~~R}$$They're widely used in the data science and machine learning communities.
It mixes the ability to execute Python (and many other supported languages including R and Julia) code with rich text-editing capabilities
Allows you to break up long experiments into smaller pieces that can be executed independently
iii. TensorFlow and Keras Package Installation
We first need to make sure that our pip is the latest version. The following code will update your pip if it is not the latest one.
pip install --upgrade pip
Use the following code to install TensorFlow. It is the current stable release for CPU and GPU.
pip install tensorflow
You can use the following code to check your Kreas and TensorFlow installation. Note that your output result might be different from this note.
!pip list -v | grep "keras\|tensorflow"
keras 2.11.0 /usr/local/lib/python3.9/dist-packages pip keras-vis 0.4.1 /usr/local/lib/python3.9/dist-packages pip tensorflow 2.11.0 /usr/local/lib/python3.9/dist-packages pip tensorflow-datasets 4.8.2 /usr/local/lib/python3.9/dist-packages pip tensorflow-estimator 2.11.0 /usr/local/lib/python3.9/dist-packages pip tensorflow-gcs-config 2.11.0 /usr/local/lib/python3.9/dist-packages pip tensorflow-hub 0.12.0 /usr/local/lib/python3.9/dist-packages pip tensorflow-io-gcs-filesystem 0.30.0 /usr/local/lib/python3.9/dist-packages pip tensorflow-metadata 1.12.0 /usr/local/lib/python3.9/dist-packages pip tensorflow-probability 0.19.0 /usr/local/lib/python3.9/dist-packages pip
You can check how many GPU(s) are available for you using the following code. Note that your output might be differernt from this note.
import tensorflow as tf
import numpy as np
print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))
Num GPUs Available: 10
Tensors are multi-dimensional arrays with a uniform type (called a dtype). You can see all supported dtypes at tf.dtypes.DType.
All tensors are *immutable, which means that you can never update the contents of a tensor*, only create a new one.
Here is a "scalar" or "rank-0" tensor . A scalar contains a single value, and no "axes".
# This will be an int32 tensor by default; see "dtypes" below.
rank_0_tensor = tf.constant(4)
print(rank_0_tensor)
tf.Tensor(4, shape=(), dtype=int32)
A "vector" or "rank-1" tensor is like a list of values. A vector has one axis:
# Let's make this a float tensor.
rank_1_tensor = tf.constant([2.0, 3.0, 4.0])
print(rank_1_tensor)
tf.Tensor([2. 3. 4.], shape=(3,), dtype=float32)
A "matrix" or "rank-2" tensor has two axes:
# If you want to be specific, you can set the dtype (see below) at creation time
rank_2_tensor = tf.constant([[1, 2],
[3, 4],
[5, 6]], dtype=tf.float16)
print(rank_2_tensor)
tf.Tensor( [[1. 2.] [3. 4.] [5. 6.]], shape=(3, 2), dtype=float16)
A scalar, shape: [] |
A vector, shape: [3] |
A matrix, shape: [3, 2] |
|---|---|---|
|
|
|
Tensors may have more axes; here is a tensor with three axes:
# There can be an arbitrary number of
# axes (sometimes called "dimensions")
rank_3_tensor = tf.constant([
[[0, 1, 2, 3, 4],
[5, 6, 7, 8, 9]],
[[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29]],])
print(rank_3_tensor)
tf.Tensor( [[[ 0 1 2 3 4] [ 5 6 7 8 9]] [[10 11 12 13 14] [15 16 17 18 19]] [[20 21 22 23 24] [25 26 27 28 29]]], shape=(3, 2, 5), dtype=int32)
There are many ways you might visualize a tensor with more than two axes.
|
|
|
You can convert a tensor to a NumPy array either using np.array or the tensor.numpy method:
np.array(rank_2_tensor)
array([[1., 2.],
[3., 4.],
[5., 6.]], dtype=float16)
rank_2_tensor.numpy()
array([[1., 2.],
[3., 4.],
[5., 6.]], dtype=float16)
Tensors often contain floats and ints, but have many other types, including:
complex numbers
strings
The base tf.Tensor class requires tensors to be "rectangular"---that is, along each axis, every element is the same size. However, there are specialized types of tensors that can handle different shapes:
Ragged tensors
Sparse tensors
a = tf.constant([[1, 2],
[3, 4]])
b = tf.constant([[1, 1],
[1, 1]]) # Could have also said `tf.ones([2,2])`
print(tf.add(a, b), "\n")
print(tf.multiply(a, b), "\n")
print(tf.matmul(a, b), "\n")
tf.Tensor( [[2 3] [4 5]], shape=(2, 2), dtype=int32) tf.Tensor( [[1 2] [3 4]], shape=(2, 2), dtype=int32) tf.Tensor( [[3 3] [7 7]], shape=(2, 2), dtype=int32)
print(a + b, "\n") # element-wise addition
print(a * b, "\n") # element-wise multiplication
print(a @ b, "\n") # matrix multiplication
tf.Tensor( [[2 3] [4 5]], shape=(2, 2), dtype=int32) tf.Tensor( [[1 2] [3 4]], shape=(2, 2), dtype=int32) tf.Tensor( [[3 3] [7 7]], shape=(2, 2), dtype=int32)
Tensors are used in all kinds of operations (or "Ops").
c = tf.constant([[4.0, 5.0], [10.0, 1.0]])
# Find the largest value
print(tf.reduce_max(c))
# Find the index of the largest value
print(tf.math.argmax(c))
# Compute the softmax
print(tf.nn.softmax(c))
tf.Tensor(10.0, shape=(), dtype=float32) tf.Tensor([1 0], shape=(2,), dtype=int64) tf.Tensor( [[2.6894143e-01 7.3105860e-01] [9.9987662e-01 1.2339458e-04]], shape=(2, 2), dtype=float32)
Note: Typically, anywhere a TensorFlow function expects a Tensor as input, the function will also accept anything that can be converted to a Tensor using tf.convert_to_tensor. See below for an example.
tf.convert_to_tensor([1,2,3])
<tf.Tensor: shape=(3,), dtype=int32, numpy=array([1, 2, 3], dtype=int32)>
tf.reduce_max([1,2,3])
<tf.Tensor: shape=(), dtype=int32, numpy=3>
tf.reduce_max(np.array([1,2,3]))
<tf.Tensor: shape=(), dtype=int64, numpy=3>