NumPy
It is the entry point from Python for DS. It is Numerical Python Library.
- How to Install:
>>> pip install numpy
- Check Version:
>>> import numpy as np
>>> np.__version__
What is the Need of NumPy?
- To perform complex mathematical operations in Data Science, ML, DL, and AI.
- NumPy defines several functions to perform complex mathematical operations.
- Performance is faster than python. Because most of the NumPy is implemented in C - Language.
- ndarray(): N - dimentional array or numpy array. Used to store large data in nd array for matplot graph style purpose.
- Used for Data Analysis.
Q: Print 10 x 10 zero matrix? (It returns float value numbers)
Ans:>>> a = np.zeros((10, 10))
>>> a
Q: Print 1 to 100 in a list?
Ans:>>> a = np.arange(1, 101)
>>> a
Q: Reshape "a" into 10 x 10 matrix? (Into 2-dimention array)
Ans:>>> a.reshape(10, 10)
Q: Print a identity matrix?
Ans:>>> a = np.identity(3)
>>> a
Q: Convert values into integer, If you want only integer numbers?
Ans:>>> a = np.zeros((10, 10), dtype = int)
>>> a
History
- Origin of NumPy is "Numeric Library" Developed by Jim Hugunin in 1995.
- Travis Oliphant merged the best features of Numeric and Numarray to create NumPy in 2005.
- It is an Open Source Library and FreeWare(available for free).
- Arrays are objects of "ndarray" class present in NumPy module.
- Array: An indexed collection of homogenious elements.
- 1-D Array: Vector
- 2-D Array: Matrix
- n-D Array: No name
Q: In which language NumPy was written?
Ans:C and Python Language.
Q: What is "ndarray" in NumPy?
Ans:The fundamental data type to store our data. It is a one class.
>>> a = np.identity(3)
>>> a
array([[1., 0., 0.],
[0., 1., 0.],
[0., 0., 1.]])
>>> type(a)
<class 'numpy.ndarray'>
Array Creation
Example:>>> a = np.ones((3, 3), dtype = int)
>>> a
array([[1, 1, 1],
[1, 1, 1],
[1, 1, 1]])
>>> np.ones((10))
array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])
Arrays can be created by 2 ways
- By using array module (Not recommended). Example:
- By using NumPy Module. Example:
>>> import array
>>> a = array.array('i', [10, 20, 30]) #i represents type:int array
>>> print(a, type(a))
array('i', [10, 20, 30]) <class 'array.array'>
Note:
Array module is not recommended because much library support is not available.
import numpy as np
>>> a = np.array([10, 20, 30])
>>> print(type(a))
<class 'numpy.ndarray'>
>>> print(a)
[10 20 30]
Attributes
Q: How to access elements of array?
- Basic indexing.
- Slice Operations.
- Advanced indexing.
- Condition based selection.
Q: Find the numbers which is divisible by 6 from an array?(It is an codition based array)
Ans:>>> a = np.arange(1, 21)
>>> a
array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20])
>>> a[a % 6 == 0]
array([ 6, 12, 18])
Q: How to iterate elements of an array?
- Ans:
- Python's normal loop.
- nditer()
- ndenumerate()
Q: In a classroom for boys fees is $3 and for girls fees is $8, For a certain batch 2200 people attended and $10100 fee collected. How many boys and girls attended for that batch?
Hints:x = No. of boys
y = No. of girls
x + y = 2200
3x + 8y = 10100
Ans:
Coefficient Matrix:
a = 1 1
3 8
>>> a = np.array([[1, 1], [3, 8]])
>>> a
array([[1, 1],
[3, 8]])
>>> b = np.array([2200, 10100])
>>> b
array([ 2200, 10100])
>>> c = np.linalg.solve(a, b)
>>> c
array([1500., 700.])
Python's List Vs NumPy ndarray
Similarityies:
- Both are used to store data.
- The order will be preserved in both. Hence indexing and slicing concepts are applicable.
- Both are mutable, i.e we can change the content.
Differences:
- List is Python's inbuilt type. But We have to install and import "numpy" explicitly.
- List can contain heterogeneous elements. But array contains only homogeneous elements. Example:
- On list we cann't perform vector operations.But on ndarray we can perform vector opeartions. Example:
- Arrays consume less memory than the List.
- Arrays are superfast when compared with List.
- NumPy arrays are more convenient to use while performing complex mathematical operations.
import numpy as np
>>> l = [10, 20.5, 'Tony', True]
>>> print(l)
[10, 20.5, 'Tony', True]
>>> a = np.array(l)
>>> print(a)
['10' '20.5' 'Tony' 'True']
import numpy as np
>>> l = [10, 20, 30]
>>> a = np.array(l)
>>> l + 2 #Invalid
>>> a + 2 #array([12, 22, 32])
>>> l / 2 #Invalid
>>> a / 2 #array([ 5., 10., 15.])
>>> l * 2 #[10, 20, 30, 10, 20, 30]
>>> a * 2 #array([20, 40, 60])
How to create NumPy Arrays
- array()
- arange()
- linespace()
- zeros()
- ones()
- full()
- eye()
- identity()
- empty()
- numpy.random
- randint()
- rand()
- uniform()
- randn()
- normal()
- shuffle()
Creation of NumPy arrays by using array():
For the given list or tuple.
>>> import numpy as np
>>> help(np.array)
1-D Array:
>>> l = [10, 20, 30]
>>> type(l)
<class 'list'>
>>> a = np.array(l)
>>> a #array([10, 20, 30])
>>> a.ndim #1
>>> type(a) #<class 'numpy.ndarray'>
>>> a.dtype #dtype('int64')
Note:
- a.ndim -- To know the dimention of ndarray.
- a.dtype -- To know the data type of elements.
2-d Array:
[[10, 20, 30], [40, 50, 60], [70, 80, 90]] -- Nested list
Example:
>>> a = np.array([[10, 20, 30], [40, 50, 60], [70, 80, 90]])
>>> a
array([[10, 20, 30],
[40, 50, 60],
[70, 80, 90]])
>>> a.ndim #2
>>> type(a) #<class 'numpy.ndarray'>
>>> a.shape #(3, 3)
>>> a.size #9
To create 1-D Array from the Tuple
>>> a = np.array(('Tony', 'Peter', 'Steve'))
>>> type(a) #<class 'numpy.ndarray'>
>>> a.ndim #1
>>> a.shape #(3,)
>>> a #array(['Tony', 'Peter', 'Steve'], dtype='<U5')
Note:
Array contains only homogeneous elements.
If the list contains heterogeneous elements: Upcasting will be performend. Int to Float.
Example:
>>> a = np.array([10, 20, 10.5])
>>> a #array([10. , 20. , 10.5]) Upcasting int to float
>>> a = np.array([10, 20, 'a'])
>>> a #array(['10', '20', 'a'], dtype='<U21')
How to create a particular Type
We have to use dtype parameter
Example:
>>> a = np.array([10, 20, 30.5], dtype = int)
>>> a #array([10, 20, 30])
>>> a = np.array([10, 20, 30.5], dtype = bool)
>>> a #array([ True, True, True])
>>> a = np.array([10, 20, 30.5], dtype = str)
>>> a #array(['10', '20', '30.5'], dtype='<U4')
>>> a = np.array([10, 20, 30.5], dtype = float)
>>> a #array([10. , 20. , 30.5])
>>> a = np.array([10, 20, 30.5], dtype = complex)
>>> a #array([10. +0.j, 20. +0.j, 30.5+0.j])
>>> a = np.array([10, "Tony"], dtype=int) #Invalid
How to create Object type Array
Example:
>>> a = np.array([10, 'Tony', True, 10.5, 10+3j], dtype=object)
>>> a #array([10, 'Tony', True, 10.5, (10+3j)], dtype=object)
>>> a = np.array([10, 'Tony', True, 10.5, 10+3j])
>>> a #array(['10', 'Tony', 'True', '10.5', '(10+3j)'], dtype='<U64')
Creation of ndarray by using arange() function
Syntax: arange([start,] stop[, step,], dtype=None, *, device=None, like=None)
Q: Create 1-D array 0 to 9?
Ans:>>> a = np.arange(10)
>>> a #array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> a.ndim #1
>>> a.shape #(10,)
>>> a.dtype #dtype('int64')
>>> a = np.arange(1, 11, 2)
>>> a #array([1, 3, 5, 7, 9])
>>> a = np.arange(1, 11, 2, dtype = float)
>>> a #array([1., 3., 5., 7., 9.])
linspace()
- In the specified interval, linearly spaced values. Syntax:
- Return evenly spaced numbers over a specified interval. Example:
linspace(start, stop, num=50, endpoint=True, retstep=False, dtype=None, axis=0, *, device=None)
>>> np.linspace(0, 1, 2) #array([0., 1.])
>>> np.linspace(0, 1, 3) #array([0. , 0.5, 1. ])
>>> np.linspace(0, 1, 4) #array([0. , 0.33333333, 0.66666667, 1. ])
>>> np.linspace(0, 1)
>>> np.linspace(0, 1).size #50
>>> np.linspace(0, 1).shape #(50,)
>>> np.linspace(1, 100, 10, dtype = int) #array([ 1, 12, 23, 34, 45, 56, 67, 78, 89, 100])
(1 - 12 = 11 numbers
12 - 23 = 11 numbers
equally spaced values)
arange() vs linspace()
- arange(): Elements will be considered in the given range based on step value.
- linspace(): The specified number of values will be considered in the given range.
zeros()
- (10, ) - 1-D array contains 10 elements.
- (5, 2) - 2-D array contains 5 rows and 2 columns. It means collectionof 1-D arrays.
- (2, 3,4) - 3-D array collection of two 2-D arrays. Syntax:
-
Example:
>>> np.zeros(4) #array([0., 0., 0., 0.])
>>> np.zeros((4, 3)) #array([[0., 0., 0.], [0., 0., 0.], [0., 0., 0.], [0., 0., 0.]])
>>> np.zeros((2, 3, 4)) #array([[[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.]], [[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.]]]) - It is used when perform some operations the result we have to store somewhere.
zeros(shape, dtype=float, order='C', *, like=None)
ones()
- Exactly same as zeros() except that instead of zero array filled with 1.
fill_value: 1
Syntax:
- Return a new array of given shape and type, filled with ones. Example:
ones(shape, dtype = None, order = 'C', *, like = None)
>>> np.ones(10)
#array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])
>>> np.ones((4, 2))
#array([[1., 1.],
[1., 1.],
[1., 1.],
[1., 1.]])
>>> np.ones((2, 3, 4), dtype = int)
#array([[[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1]],
[[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1]]])
full()
- It is used to create a array. Syntax:
- Returb a new array of given shape and type, filled with 'fill_value' Example:
full(shape, fill_value, dtype = None, order = 'C', *, like = None)
>>> np.full(10)
#TypeError: full() missing 1 required positional argument: 'fill_value'
>>> np.full(10, 2) #array([2, 2, 2, 2, 2, 2, 2, 2, 2, 2])
>>> np.full(10, 3) #array([3, 3, 3, 3, 3, 3, 3, 3, 3, 3])
>>> np.full((5, 4), 3)
#array([[3, 3, 3, 3],
[3, 3, 3, 3],
[3, 3, 3, 3],
[3, 3, 3, 3],
[3, 3, 3, 3]])
>>> np.full((2, 3, 4), 9)
#array([[[9, 9, 9, 9],
[9, 9, 9, 9],
[9, 9, 9, 9]],
[[9, 9, 9, 9],
[9, 9, 9, 9],
[9, 9, 9, 9]]])
Note: >>> np.full(shape=(3, 4), fill_value = 6)
#array([[6, 6, 6, 6],
[6, 6, 6, 6],
[6, 6, 6, 6]])
>>> np.full((3, 4), fill_value=6)
#array([[6, 6, 6, 6],
[6, 6, 6, 6],
[6, 6, 6, 6]])
>>> np.full(shape=(3, 4), 6)
#SyntaxError: positional argument follows keyword argument
eye()
- It create any dimention identity matrix. Syntax:
- Return a 2-D array with ones on the diagonal and zeros elsewhere. Example:
eye(N, M=None, k=0, dtype=<class 'float'>, order='C', *, device=None, like=None)
N - Number of rows
M - Number of Columns
K - Meant for diagonal
>>> np.eye(2, 3)
#array([[1., 0., 0.],
[0., 1., 0.]])
>>> np.eye(3, dtype = int)
#array([[1, 0, 0],
[0, 1, 0],
[0, 0, 1]])
>>> np.eye(3, k = 1 , dtype = int)
#array([[0, 1, 0],
[0, 0, 1],
[0, 0, 0]])
identity()
- It is exactly same as "eye()" function except that, it is always square matrix(the number of rows and number of columns always same).
- Only main diagonal contains 1's.
- identity() function is a special case of eye() function. Syntax:
- Return the identity array. The identity array is a square array with ones on the main diagonal. Example:
identity(n, dtype=None, *, like=None)
>>> np.identity(3)
#array([[1., 0., 0.],
[0., 1., 0.],
[0., 0., 1.]])
>>> np.identity(3, dtype = int)
#array([[1, 0, 0],
[0, 1, 0],
[0, 0, 1]])
empty()
- One empty array will be created. Garbege value printed. Syntax:
- Return a new array of given shape and type, without initializing entries. Example:
empty(shape, dtype=float, order='C', *, device=None, like=None)
>>> np.empty(3)
#array([1., 1., 1.])
>>> np.empty((3, 3))
#array([[5.e-324, 0.e+000, 0.e+000],
[0.e+000, 5.e-324, 0.e+000],
[0.e+000, 0.e+000, 5.e-324]])
zeros() vs empty()
- If we required an array only with zeros then we should go for zeros().
- If we never worry about data, just we required an empty array for future purpose, then we should go for empty().
- The time required to create empty array is very very less when compared with zeros array. i.e performance wise empty function is recommended than zeros if we are not worry about data.
Performance comparision of zeros() and empty()
Program:
import numpy as np
from datetime import datetime
begin = datetime.now()
a = np.zeros((10000, 300, 400))
after = datetime.now()
print("Time taken by zeros: ", after-begin)
a = None
begin = datetime.now()
a = np.empty((10000, 300, 400))
after = datetime.now()
print("Time taken by empty: ", after-begin)
Array Creation by using random module
- randint()
- rand()
- uniform()
- randn()
- normal()
- shuffle()
randint():
- To generate a random integer value in the given range. Example:
- Return random integers from `low` (inclusive) to `high` (exclusive).
[low, high)
Example:
- To create 1-D array of size 10 with random values from 1 to 9.
- 2-D Array from 0 to 99 random values
- 3-D Array
- We have to use astype() method. Example:
>>> import numpy as np
>>> help(np.randint) (#Error)
>>> help(np.random.randint)
Syntax:
randint(low, high=None, size=None, dtype=int)
>>> np.random.randint(10, 20)
>>> np.random.randint(1, 10, size = 10)
#array([9, 3, 6, 7, 5, 5, 8, 7, 4, 8], dtype=int32)
>>> np.random.randint(100, size = (3, 4))
>>> np.random.randint(100, size = (2, 3, 4))
How to convert from one array type to another type
>>> a = np.random.randint(1, 10, size = 10)
>>> a.dtype
#dtype('int32')
>>> b = a.astype('float')
>>> b.dtype
#dtype('float64')
rand():
- Uniform distribution: -- 10 11 9 10 11 10
- Normal distribution: -- 6 4 10 4 14 (10 is mean value)
- It will generate random float values in the range(0, 1) from uniform distribution samples. Example:
- 1-D Array: Example:
- 2-D Array: Example:
- 3-D Array: Example:
>>> np.random.rand() #A single float value will be generated.
>>> np.random.rand(10)
>>> np.random.rand(2, 3)
>>> np.random.rand(2, 3, 4)
uniform():
- rand(): Range is always [0, 1]
- uniform(): Customized range. Syntax:
- 1-D Array:
- 2-D Array:
- 3-D Array:
uniform(low=0.0, high=1.0, size=None)
Example:
>>> np.random.uniform()
>>> np.random.uniform(10, 20)
>>> np.random.uniform(10, 20, size = 5)
>>> np.random.uniform(10, 20, size = (3, 4))
>>> np.random.uniform(10, 20, size = (2, 3, 4))
randn():
- Values from normal distribution with mean 0 and variance is 1.
- 1-D Array:
- 2-D Array:
- 3-D Array:
>>> np.random.randn(10)
>>> np.random.randn(2, 3)
>>> np.random.randn(2, 3, 4)
normal():
- We can customize mean and variance. Syntax:
- 1-D Array:
- 2-D Array:
- 3-D Array:
normal(loc=0.0, scale=1.0, size=None)
>>> np.random.normal(10, 4, size = 10)
>>> np.random.normal(10, 4, size = (2, 3))
>>> np.random.normal(10, 4, size = (2, 3, 4))
shuffle():
- Modify a sequence in-place by shuffling its contents.
- 1-D Array:
- 2-D Array: Internal content never changed only the rows shuffled.
- 3-D Array: If we shuffle for 3-D array, then the order of 2-D arrays will be changed but not it's internal content.
>>> a = np.arange(9)
>>> a
#array([0, 1, 2, 3, 4, 5, 6, 7, 8])
>>> np.random.shuffle(a) (inline shuffling happens)
>>> a
#array([4, 2, 6, 1, 5, 8, 3, 7, 0])
>>> a = np.random.randint(1, 101, size = (6, 5))
>>> a
>>> np.random.shuffle(a)
>>> a
>>> a = np.arange(48).reshape(4, 3, 4)
>>> a
>>> np.random.shuffle(a)
>>> a
Summary of random library functions
- randint(): To generate random int values in the given range.
- rand(): To generate uniform distribution float values in the range of [0, 1)
- uniform(): To generate uniform distributed float values in the given range. [low, high]
- randn(): Normal distributed float values with mean value 0 and standard deviation 1.
- normal(): Normal distributed float values with specified mean and standard deviation.
- shuffle(): To shuffle order of elements in the given array.
Array Attributes
- ndim: -- Returns the dimension of the array.
- shape: -- Returns the shape of the array (10,) 1-D, (10, 3) 2-D
- size: -- To get total number of elements.
- dtype: -- To get data type of elements of the array.
- itemsize: -- Lenght of each element of array in bytes (4 - bytes) Example:
>>> a = np.array([10, 20, 30, 40])
>>> a.ndim
#1
>>> a.shape
#(4,)
>>> a.size
#4
>>> a.dtype
#dtype('int64')
>>> a.itemsize
#8
NumPy Data Types
- Python Data Types: int, float, str, complex, bool, etc...
- NumPy Data Types: Multiple data types present (Python + C).
- i -- integer (int8, int16, int 32, int64)
- b -- boolean
- u -- unsigned integer (uint8, uint16, uint32, uint64)
- f -- float (float16, float32, float64)
- c -- complex (complex64, complex128)
- s -- string
- U -- Unicode String
- M -- datetime etc...
- int8 -- i1, int16 -- i2, int32 -- i4(default)
- float16 -- f2, float32 -- f4(default), float64 -- f8
- int8:
- The value will be represnted by 8bits.
- MSB (Most Significant Bits) is reserved for sign.
- The range: -128 to 127
Changing the data type of an existing array
- "astype()" Ex:
- By using built-in function of NumPy like "float64" Ex:
- Boolean: Ex:
>>> import numpy as np
>>> a = np.array([10, 20, 30])
>>> b = a.astype('float64')
>>> print(a.dtype)
#int64
>>> b.dtype
#dtype('float64')
>>> a = np.array([10, 20, 30])
>>> b = np.float64(a)
>>> a.dtype
#dtype('int64')
>>> b.dtype
#dtype('float64')
>>> a = np.array([0, 20, 0])
>>> b = np.bool(a) #Invalid
>>> b = np.bool_(a)
>>> b
#array([False, True, False])
How to get/access elements of NumPy Array
- Indexing
- Slicing
- Advanced Indexing
Indexing:
- By using index, we can get single element of the array.
- Zero based indexing. i.e the index of the first element is 0.
- Supports both +ve and -ve indexing.
From 1-D Array:
- a[index] Ex:
>>> a = np.array([10, 20, 30, 40])
>>> a
#array([10, 20, 30, 40])
>>> a[-1] #40
From 2-D Array:
- a[rowIndex][columnIndex] Ex:
- To access 50
>>> a = np.array([[10, 20, 30], [40, 50, 60]])
>>> a
#array([[10, 20, 30],
[40, 50, 60]])
>>> a[1][1] #50
>>> a[1][-2]
>>> a[-1][-2]
>>> a[-1][1]
From 3-D Array:
- a[i][j][k]
- i -- Represents which 2-D Array(index of 2-D Array),
j -- Represents row index in that 2-D array
k -- Represents column index of that 2-D array - a[0][1][2]:
- 0-indexed 2-D array.
- In that 2-D array indexed row and 2 indexed column.
Ex:
- To access 14
>>> l = [[[1, 2, 3], [4, 5, 6], [7, 8, 9]], [[10, 11, 12], [13, 14, 15], [16, 17, 18]]]
>>> l
#[[[1, 2, 3], [4, 5, 6], [7, 8, 9]], [[10, 11, 12], [13, 14, 15], [16, 17, 18]]]
>>> a = np.array(l)
>>> a
#array([[[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9]],
[[10, 11, 12],
[13, 14, 15],
[16, 17, 18]]])
>>> a[1][1][1]
>>> a[-1][-2][-2]
>>> a[1][-2][-2]
>>> a[-1][1][-2]
Accessing elements of ndarray by using slice operator
- 1-D Array: a[begin:end:step]
- 2-D Array: a[row, column], a[begin:end:step, begin:end:step]
- Access [10, 20]
- To use slice operator, compulsory elements should be in order. We cann't select elements which are out of order. i.e we cann't select arbitrary elements.
>>> a = np.arange(10, 101, 10)
>>> a
#array([ 10, 20, 30, 40, 50, 60, 70, 80, 90, 100])
>>> a[2:5]
#array([30, 40, 50])
>>> a[::1]
#array([ 10, 20, 30, 40, 50, 60, 70, 80, 90, 100])
>>> a[::-1]
#array([100, 90, 80, 70, 60, 50, 40, 30, 20, 10])
>>> a[::-2]
#array([100, 80, 60, 40, 20])
>>> a = np.array([[10, 20], [30, 40], [50, 60]])
>>> a
#array([[10, 20],
[30, 40],
[50, 60]])
>>> a[0:1]
#array([[10, 20]])
>>> a[0:1, :]
#array([[10, 20]])
>>> a[0, :]
#array([10, 20]) #It is 1-D array
>>> a[0::2, :]
#array([[10, 20],
[50, 60]])
>>> a[0: 2, 1:2]
#array([[20],
[40]])
>>> a[:2, 1:]
Ex:
>>> a = np.array([[1, 2, 3, 4], [5, 6, 7,8], [9, 10, 11, 12], [13, 14, 15, 16]])
>>> a
#array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]])
>>> a[:2, :]
#array([[1, 2, 3, 4],
[5, 6, 7, 8]])
>>> a[0::3, :]
#array([[ 1, 2, 3, 4],
[13, 14, 15, 16]])
>>> a[:, :2]
#array([[ 1, 2],
[ 5, 6],
[ 9, 10],
[13, 14]])
>>> a[:, :: 2]
#array([[ 1, 3],
[ 5, 7],
[ 9, 11],
[13, 15]])
>>> a[1:3, 1:3]
#array([[ 6, 7],
[10, 11]])
>>> a[0::3, 0::3]
#array([[ 1, 4],
[13, 16]])
Slice in 3-D Array
(2, 3, 4) -- (i, j, k)
2 -- Number of 2-D Arrays
3 -- The number of rows
4 -- Number of columns
>>> a[i, j, k]
>>> a[begin:end:step, begin:end:step, begin:end:step]
Ex:
>>> import numpy as np
>>> l = [[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]], [[13, 14, 115, 16], [17, 18, 19, 20], [21, 22, 23, 24]]]
>>> a = np.array(l)
>>> a
>>> a[:, :, :1]
#array([[[ 1],
[ 5],
[ 9]],
[[13],
[17],
[21]]])
>>> a[:, :1, :]
#array([[[ 1, 2, 3, 4]],
[[ 13, 14, 115, 16]]])
>>> a[:, ::2, :]
#array([[[ 1, 2, 3, 4],
[ 9, 10, 11, 12]],
[[ 13, 14, 115, 16],
[ 21, 22, 23, 24]]])
>>> a[:, :2, 1:3]
#array([[[ 2, 3],
[ 6, 7]],
[[ 14, 115],
[ 18, 19]]])
>>> a[:, ::3, ::3]
#array([[[ 1, 4],
[ 9, 12]],
[[13, 16],
[21, 24]]])
Advanced Indexing
- By usnig index, we can access only one element at a time.
- By using slice operator we can access multiple elements at a time, but all elements should be in order/sequence.
- 1-D Array:
- array[x]: "x" can be either nd array or list, which represents required indexes.
- 1st Way: Required values are "30, 50, 60, 90":
- Create ndarray with required indices.
- Pass this indices array as argument to orignal array
- 2nd Way:
- We can select elements based on some condition also. Syntax:
- In the boolean array, where ever True present the corresponding value will be selected. Ex:
- Select elements which are greater than 25
1-D -- a[i]
2-D -- a[i][j]
3-D -- a[i][j][k]
1-D -- a[begin:end:step]
2-D -- a[begin:end:step, begin:end:step]
3-D -- a[begin:end:step, begin:end:step, begin:end:step]
Accessing Multiple arbitrary elements
>>> a = np.arange(10, 101, 10)
>>> a
#array([ 10, 20, 30, 40, 50, 60, 70, 80, 90, 100])
>>> indices = np.array([2, 4, 5, 8])
>>> indices
#array([2, 4, 5, 8])
>>> a[indices]
#array([30, 50, 60, 90])
>>> l = [2, 4, 5, 8]
>>> a[l]
#array([30, 50, 60, 90])
Required value: [10, 50, 70, 100]
>>> l = [0, 4, 6, 9]
>>> a[l]
#array([ 10, 50, 70, 100])
>>> indices = np.array([0, 4, 6, 9])
>>> a[indices]
Values: [10, 100, 50, 70]
>>> l = [0, 9, 4, 6]
>>> a[l]
#array([ 10, 100, 50, 70])
>>> a[[0, 9, 4, 6]]
value: [10, 100]
>>> a[[0, 9]]
#array([ 10, 100])
>>> a[[0, -1]]
Access elements of 2-D Array:
>>> l = [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]]
>>> l
#[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]]
>>> a = np.array(l)
>>> a
#array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]])
Syntax:
a[[row_indices], [column_indices]]
Ex: Value: [1, 6, 11, 16]
>>> a[[0, 1, 2,3], [0, 1, 2, 3]]
#array([ 1, 6, 11, 16])
It select elements from: (0, 0), (1, 1), (2, 2), (3, 3)
Ex: Value: [2, 8, 9, 15]
>>> a[[0, 1, 2, 3], [1, 3, 0, 2]]
#array([ 2, 8, 9, 15])
L-Shape Elements:
>>> a[[0, 1, 2, 3, 3, 3, 3], [0, 0, 0, 0, 1, 2, 3]]
#array([ 1, 5, 9, 13, 14, 15, 16])
Accessing multiple arbitrary elements in 3-D array:
-
Ex:
>>> a = np.arange(1, 25).reshape(2, 3, 4)
>>> a
array([[[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]],
[[13, 14, 15, 16],
[17, 18, 19, 20],
[21, 22, 23, 24]]])
Syntax:
a[[indices of 2-D array], [row indices], [column indices]]
Accessing 7 and 18 from the array
>>> a[[0, 1], [1, 1], [2, 1]]
#array([ 7, 18])
Condition based selection:
array[boolean_array]
>>> a = np.array([10, 20, 30, 40])
>>> boolean_array = np.array([True, False, False, True])
>>> boolean_array
#array([ True, False, False, True])
>>> a[boolean_array]
#array([10, 40])
>>> b_a = a > 25 (#first Way)
>>> a[b_a]
#array([30, 40])
>>> a[a > 25] (#2nd way)
#array([30, 40])
Ex:
>>> a = np.array([10, -5, 20, 40, -3, -1, 75])
>>> a
#array([10, -5, 20, 40, -3, -1, 75])
>>> a[a < 0]
#array([-5, -3, -1])
>>> a[a > 0]
#array([10, 20, 40, 75])
>>> a[a % 2 == 0]
#array([10, 20, 40])
Condition Based selection 2-D Array also
>>> a = np.arange(1, 26).reshape(5, 5)
>>> a
#array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25]])
>>> a[a % 2 == 0]
#array([ 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24])
>>> a[a % 10 == 0]
#array([10, 20])
Slicing vs Advanced Indexing
- In case of list, slice operator will creates a separate copy.
- if we perform any changes in one copy those changes won't be reflected in other copy. Ex:
- A separate copy won't be created and just we are getting view of the original copy.
Python's Slicing:
l1 = [10, 20, 30, 40]
l2 = l1[:]
l2[1] = 333
l[1] = 999
print(l1)
print(l2)
NumPy Array Slicing:
>>> a = np.arange(10, 101, 10)
>>> a
>>> b = a[0:4]
>>> b
>>> a[0] = 333
>>> a
>>> b
b[1] = 999
b
a
Advanced Indexing and Condition Based Selection
- It will select required element based on provided index or condition and with those elements a new 1-D array object will be created.
- The output is always a new 1-D array only. Ex:
>>> a = np.arange(10, 101, 10)
>>> a
#array([ 10, 20, 30, 40, 50, 60, 70, 80, 90, 100])
>>> b = a[[0, 2, 5]]
>>> b
#array([10, 30, 60])
>>> a[0] = 333
>>> a
#array([333, 20, 30, 40, 50, 60, 70, 80, 90, 100])
>>> b
#array([10, 30, 60])
>>> b[0] = 999
>>> b
#array([999, 30, 60])
>>> a
#array([333, 20, 30, 40, 50, 60, 70, 80, 90, 100])
Slicing Vs Advanced Indexing
Slicing:- The elements should be ordered.
- We can't select arbitrary elements.
- Conditional based selection is not possible.
- Just we will get view but not copy.
- Memory, performance-wise it is the best.
- The elements need not be ordered.
- We can select arbitrary elements.
- Condition based selection is possible.
- Just we will get separate copy but not view.
- Memory, performance-wise not up to mark.
Summary of Syntaxes
Basic Indexing:- 1-D Array:
a[i]
- 2-D Array:
a[i][j] or a[i, j]
- 3-D Array:
a[i][j][k] or a[i, j, k]
- 1-D Array:
a[begin: end: step]
- 2-D Array:
a[begin: end: step, begin: end: step]
- 3-D Array:
a[begin: end: step, begin: end: step, begin: end: step]
- 1-d Array:
a[x]
-- x contains ndarray or list - 2-D Array:
a[[row_indices], [column_indices]]
- 3-D Array:
a[[indices of 2-D Array], [row_indices], [column_indices]]
a[condition] -- a[a > 0]
- This is same for all 1-D, 2-D and 3-D Arrays.
How to Iterate Elements of the ndarray
Iteration means getting all elements one-by-one.
3-Ways:- By using Python's loop.
- By using nditer() function.
- By using ndenumerate() function.
- To iterate Elements of 1-D Array Ex:
- To iterate Elements of 2-D Array Ex:
- To iterate Elements of 3-D Array Ex:
import numpy as np
a = np.arange(10, 51, 10)
for x in a:
print(x)
a = np.array([[10, 20, 30], [40, 50, 60], [70, 80, 90]])
# print(a)
for x in a: # x is 1-D array but not scalar value
# print(x)
for y in x: # y is a scalar value present in 1-D array
print(y)
a = np.array([[[10, 20], [30, 40]], [[50, 60], [70, 80]]])
# print(a)
for x in a: # x is 2-D Array but not scalar value
# print(x)
for y in x: # y is 1-D Array but not scalar value
# print(y)
for z in y: # z is a scalar value
print(z)
Note:
To iterate elements of n-D Array, we required n-;oops.
- Advantages for any n-D Array only onne loop is enough.
- nditer is a class present in numpy module.
- nditer() -- Creating an object of nditer class.
1-D Array:
Ex:
a = np.arange(10, 51, 10)
for x in np.nditer(a):
print(x)
2-D Array:
Ex:
a = np.array([[10, 20, 30], [40, 50, 60], [70, 80, 90]])
for x in np.nditer(a):
print(x)
3-D Array:
Ex:
a = np.array([[[10, 20], [30, 40]], [[50, 60], [70, 80]]])
for x in np.nditer(a):
print(x)
Iterate elements of Sliced Array:
Ex:
a = np.array([[10, 20, 30], [40, 50, 60], [70, 80, 90]])
# print(a[:,:2])
for x in np.nditer(a[:, : 2]):
print(x)
Using nditer() to get elements of required datatype:
Ex:
for x in np.nditer(a, flags=['buffered'], op_dtypes=['float']):
"op_dtypes" is used to change type
Normal Python Loops vs nditer():
-
Python:
- n-loops are required.
- There is no way to specify our required dtype.
- Only one loop is enough.
- There is a way to specify required dtype. For this we have to use "op_dtype" argument.
- If we want to find co-ordinates also in addition to element.
- Array indices(co-ordinates) and values.
1-D Array:
Ex:
a = np.array([10, 20, 30, 40, 50])
for pos, element in np.ndenumerate(a):
print(f'{element} element present at index/position: {pos}')
2-D Array:
Ex:
a = np.array([[10, 20, 30], [40, 50, 60], [70, 80, 90]])
for pos, element in np.ndenumerate(a):
print(f'{element} element present at index/position: {pos}')
3-D Array:
Ex:
a = np.arange(1, 25).reshape(2, 3,4)
for pos, element in np.ndenumerate(a):
print(f'{element} element present at index/position: {pos}')
Arithmatic Operators (+, -, *, /, **, //)
1-D Array:-
Ex:
>>> a = np.array([10, 20, 30, 40])
>>> print(a + 2)
>>> print(a - 2)
>>> print(a * 2)
>>> print(a / 2)
>>> print(a % 2)
>>> print(a // 2)
- Ex:
- In python anything by
"zero"
including"zero/zero"
result is always"ZeroDivisionError"
. - But in NumPy there is no
"ZeroDivisionError"
Ex:
>>> a = np.array([[10, 20, 30], [40, 50, 60]])
>>> print(a + 2)
>>> print(a - 2)
>>> print(a * 2)
>>> print(a / 2)
>>> print(a % 2)
>>> print(a // 2)
10/0 -- infinity(inf)
0/0 -- undefined(NaN -- Not a Number)
Arithmatic Operators for Arrays with Arrays
- Compulsory both arrays should have
- Same dimention
- Same shape and
- Same size
- Otherwise we will get error.
-
Ex:
>>> a = np.array([10, 20, 30, 40])
>>> b = np.array([1, 2, 3, 4])
>>> a.ndim
# 1
>>> b.ndim
# 1
>>> a.shape
# (4,)
>>> b.shape
# (4,)
>>> a.size
# 4
>>> b.size
# 4
>>> b + a
# array([11, 22, 33, 44])
>>> b - a
# array([ -9, -18, -27, -36])
>>> a + b
# array([11, 22, 33, 44])
>>> a - b
# array([ 9, 18, 27, 36])
>>> a * b
# array([ 10, 40, 90, 160])
>>> a / b
# array([10., 10., 10., 10.])
>>> a // b
# array([10, 10, 10, 10])
-
Ex:
a = np.array([[1, 2], [3, 4]])
b = np.array([[5, 6], [7, 8]])
a + b
a - b
a * b
b / a
b // a
Ex:
(If dimention or size not same on both array we will get this error)
>>> a = np.array([10, 20, 30])
>>> b = ([10, 20, 30, 40])
>>> a + b
ValueError: operands could not be broadcast together with shapes (3,) (4,)
Equivalent function for arithmatic operators in numpy
-
Ex:
>>> a = np.array([10, 20, 30])
>>> b = np.array([1, 2, 3])
>>> np.add(a, b)
>>> np.subtract(a, b)
>>> np.multiply(a, b)
>>> np.divide(a, b)
>>> np.floor_divide(a, b)
>>> np.mod(a, b)
>>> np.power(a, b)
The functions which operates element by element on whole array are called as universal functions(ufunc).
Broadcasting
- Even though dimentions are different, shapes are different and sizes are different still some arithmatic operations are allowed by broadcasting.
- Broadcasting will be performed automatically by numpy itself and we are not required to perform explicitly.
Rules for Broadcasting
- Rule-1: Make sure both arrays should be same dimention. Padded(Add) 1's in the shape of lesser dimention array on the left side untill both array have same dimension. Ex:
- Rule-2: If the size of two arrays doesn't match in any dimention, then the arrays with size equal to 1 in that dimention will be increased to the size of other dimention to match.
- Now dimension, shapes and size are equal.
- Ex:
- Same dimention but different shapes, so numpy unable to perform broadcasting.
- The data will be reused from the same input array.
- If the rows are required then re-use existing row.
- If the columns are required then re-use existing columns.
- The result is always higher dimension of input arrays. Ex:
-
Before:
(4, 3) -- 2-D
(3,) -- 1-D
After:
(4, 3) -- 2-D
(1, 3) -- 2-D
Note:
If any dimention, the sizes are not matched and neither equal to 1, then we will get an error, NumPy does not able to perform broadcasting between those arrays.
Ex:
-
Before:
(4, 3) -- 2-D
(1, 3) -- 2-D
After:
(4, 3) -- 2-D
(4, 3) -- 2-D
Broadcasting betwwon between (3, 2, 2) and (3,) possible or not?
-
Before Rule-1:
(3, 2, 2)
(3,)
After Rule-1:
(3, 2, 2)
(1, 1, 3)
After Rule-2:
(3, 2, 2)
(3, 2, 3)
input: 3-D, 1-D
output: 3-D
>>> a = np.array([10, 20, 30])
>>> b = np.array([40])
>>> a + b
#array([50, 60, 70])
Ex:
>>> a = np.array([[10, 20], [30, 40], [50, 60]])
>>> b = np.array([10, 20])
>>> a + b
# array([[20, 40],
[40, 60],
[60, 80]])
Ex:
>>> a = np.array([[10], [20], [30]])
>>> b = np.array([10, 20, 30])
>>> a + b
# array([[20, 30, 40],
[30, 40, 50],
[40, 50, 60]])
Array Manipulation Functions
- .reshape()
- .resize()
- .flatten()
- .flat variable()
- .ravel()
- .transpose()
- .swapaxes()
.reshape():
- One shape to another shape. Ex:
- The data should not be changed.
- Input size and output size must be matched. Ex: 1
- No change in the data, New array object won't be created.
- Just we are getting view of existing object.
- View but not copy
- If we perform any changes in the original array, that change will be reflected to reshaped array viceversa. Ex:
- If we can specify unknown dimention size as -1. Ex:
-
>>> help(np.reshape)
reshape(a, newshape, order='C')- C Style -- Row major Order
- Fortran Style -- Column Major Order Ex:
>>> a = np.arange(12).reshape(3, 4)
>>> a # array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]])
>>> b = np.reshape(a, (12,), 'C')
>>> b # array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
>>> b = np.reshape(a, (12,), 'F')
>>> b # array([ 0, 4, 8, 1, 5, 9, 2, 6, 10, 3, 7, 11])
Ex:
>>> a = np.arange(24)
>>> a
>>> np.reshape(a, (6, 4), 'C')
>>> np.reshape(a, (6, 4), 'F')
(10,) -- (5, 2), (2, 5), (10, 1), (1, 10)
(24,) -- (3, 8) -- (2, 3, 4), (6, 4), (2, 2, 2, 3)
>>> a = np.arange(10)
>>> a
>>> b = np.reshape(a, (5, 2))
>>> b = np.reshape(a, (10, 1))
>>> b = np.reshape(a, (1, 5, 2)) # 3-D array 5-rows and 2-cols
Ex: 2
>>> a = np.arange(24)
>>> b = np.reshape(a, (6, 4))
>>> b = np.reshape(a, (2, 3, 4))
>>> b = np.reshape(a, (6, 5)) # ValueError: cannot reshape array of size 24 into shape (6,5)
>>> a = np.arange(12)
>>> print(a)
>>> b = np.reshape(a, (4, 3))
>>> print(b)
>>> a[0] = 333
>>> print(a)
>>> print(b)
>>> b[1][1] = 999
>>> print(a)
>>> print(b)
a = (12,)
b = (6, -1) -- (6, 2) # size considered
>>> a = np.arange(12)
>>> b = np.reshape(a, (6, 2))
>>> b
>>> b = np.reshape(a, (6, -1))
>>> b
>>> b = np.reshape(a, (-1, 3))
>>> b
>>> b = np.reshape(a, (-1, -1)) # Error
>>> a = np.arange(24)
>>> b = np.reshape(a, (2, 3, -1))
>>> b = np.reshape(a, (2, -1, 4))
>>> b = np.reshape(a, (-1, 3, 4))
>>> b = np.reshape(a, (3, 4, -1))
>>> b = np.reshape(a, (5, -1)) # Error
Conclusion:
- To reshape array without changing data.
- The size must be matched.
- We can use either numpy library function or ndarray class method.
- np.reshape()
- a.reshape()
- It won't create a new array object, just we will get view.
- We can use -1 in unknown dimention, but only once.
- Order: 'C', 'F'
.resize():
Output Array: can be any dimention, any shape, any size.
- Input size and output size need not be matched.
- The data may be changed.
- We will get copy but not view.
- How to get that new data.
- np.resize() -- Repeate elements of input array.
- a.resize() -- Use zero for extra elements.
- -1 such type of story not applicable for resize()
- input: (10,)
- reshape: (5, -1)
- resize: (5, -1)
- If we use ndarray class resize() method, inline modification will be happened. Ex:
>>> a = np.arange(1, 6)
>>> a
>>> b = np.resize(a, (4, 3))
>>> b
>>> a[0] = 333
>>> a
>>> b
>>> b[0][1] = 999
>>> b
>>> a
>>> refcheck = False
>>> a = np.arange(1, 6)
>>> a.resize(5, 3)
>>> a
Q: Difference between np.resize() and ndarray.resize()?
Ans:np.resize() | ndarray.resize() |
---|---|
It is a library function in numpy module. | It is a method present in ndarray class. |
It will create new array and returned it. | It won't return new array and existing array will be modified. |
If the new_shape required more elements then repeated copies of original array will be reused. | Extra elements filled with zeros. |
Q: Difference between reshape() and resize()?
Ans:reshape() | resize() |
---|---|
It is just to reshape to array without changing size and data. | It is to resize() array, data may be changed, size may be changed. |
Just a view will be created but not copy. If we perform any changes in the original array automatically those changes will be reflected in reshaed copy also. | Separate copy will be created. If we perform any changes in the original array those changes won't be reflected in resize array. |
We can use -1 in unknown dimention. | There is no story like -1. |
.flatten():
- It convert any n-D array to 1-D array.
- It is a method present in ndarray class but not numpy library function.
- a.flatten() -- Valid
- np.flatten() -- Invalid
- a.flatten(order='C')
- C-style -- row major order
- F-style -- column major order
- It will create a new copy and returns it. (i.e copy but not view)
- The output of the flatten method is always 1-D array. Ex:
>>> a = np.arange(6).reshape(3, 2)
>>> a
>>> a.flatten()
>>> a.flatten('F')
>>> b = a.flatten()
>>> a[0][0] = 3113
>>> a
>>> b
>>> b[1] = 999
>>> b
>>> a
Ex:
>>> a = np.arange(1, 19).reshape(3, 3, 2)
>>> a
>>> b = a.flatten()
>>> a.ndim #3
>>> b.ndim #1
>>> b
.flat variable:
- It is a 1-D iterator over the array.
- This is a 'numpy.flatiter' instance.
-
Ex:
>>> a = np.arange(1, 19).reshape(3, 3, 2)
>>> a
>>> a.flat[2] #3
>>> a.flat[10] #11
>>> for x in a.flat: print(x)
.ravel():
- Convert any n-D array to 1-D array.
- It is a method present in ndarray class and also numpy library function.
- a.ravel() -- Valid
- np.ravel() -- Valid
- a.ravel(order='C' | 'F')
- It returns view but not copy.
- The output of ravel method is always 1-D array. Ex:
>>> help(np.ravel)
>>> help(np.ndarray.ravel)
Ex:
>>> a = np.arange(24).reshape(2, 3, 4)
>>> a
>>> b = a.ravel() #view
>>> b[0] = 333
>>> b
>>> a
>>> a = np.arange(18).reshape(6, 3)
>>> a
>>> b = np.ravel(a)
>>> b
Q: Difference between faltten() and ravel()?
Ans:flatten() | ravel() |
---|---|
To convert any n-D array to 1-D array and a new array object will be created. | To convert any n-D array to 1-D array but returns just view but not copy. |
If we perform any changes in the flatten copy, then those changes won't be reflected in the original copy. | If we perform any changes in the ravel copy, then those chnages will be reflected in the original copy. |
It is ndarray class method but not numpy library function. | We can use as method ans as well as a function. |
.transpose():
>>> help(np.transpose)
- transpose(a, axes = None)
- Reverse or permute the axes of an array.
- Returns the modified array. Ex:
- No change in data hence it returns only view but not copy.
- 2 -- 2-D Arrays
- 3 -- 3 rows
- 4 -- 4 Columns
- Total size = 24
- Transpose of 1-D array will be generated same array only. Ex:
- If we are not using axes parameter, then dimention will be reversed.
- Axes parameter describes in which order we have to take axes.
- It is helpful for 3-D and 4-D arrays. For 3-D array: (2, 3, 4)
- The size of axis - 0: 2
- The size of axis - 1: 3
- The size of axis - 2: 4
- For 1-D array, there is no impact of transpose() function.
- If we are not using axes argument, then dimentions will be reversed.
- If we provide axes argument, then we can specify our own order axes.
- Repeated axis in transpose is not allowed (0, 2, 2).
- Axes argument is more helpful from 3-D array oneards but not for 2-D array.
- Various possible syntaxes:
- np.transpose(a)
- np.transpose(a, axes=(2, 0, 1))
- ndarray.transpose()
- ndarray.T
- ndarray.transpose(*axes)
>>> a = np.arange(1, 5).reshape(2, 2)
>>> a
>>> b = np.transpose(a)
>>> b
Note:
>>> a[0][0] = 333
>>> a
>>> b
For 3-D Array: (2, 3, 4)
transpose(a, axes = None)
transpose(a) -- (4, 3, 2)
Ex:
>>> a = np.arange(24).reshape(2, 3, 4)
>>> a
>>> b = np.transpose(a)
>>> b
>>> b.shape #(4, 3, 2)
>>> a = np.arange(6)
>>> a
>>> np.transpose(a)
- Ex:
a = (2, 3, 4, 5)
transpose(a) -- (5, 4, 3, 2)
axes parameter:
np.transpose(a) -- (4, 3, 2)
Q: My required order is: (2, 4, 3) / (4, 2, 3) / (3, 4, 2)
np.transpose(a, axes = (0, 2, 1) / (2, 0, 1) / (1, 2, 0))
np.transpose(a, axes=(0, 2, 2)) #ValueError: repeated axis in transpose
ndarray class transpose() method:
Ex:
>>> help(np.ndarray.transpose)
a.transpose(*axes)
Returns a view of the array with axes transposed.
Ex:
>>> a = np.arange(24).reshape(2, 3, 4)
>>> a.shape #(2, 3, 4)
>>> b = a.transpose()
>>> b.shape #(4, 3, 2)
>>> b = a.transpose((2, 0, 1))
>>> b.shape #(4, 2, 3)
>>> b = a.T
>>> b.shape #(4, 3, 2)