Blog

Machine Learning is a technique of data analysis that combines data with statistical tools to predict the output. This prediction is used by the various corporate industries to make a favorable decision.

PySpark provides an API to work with the Machine learning called as mllib. PySpark’s mllib supports various machine learning algorithms like classification, regression clustering, collaborative filtering, and dimensionality reduction as well as underlying optimization primitives. Various machine learning concepts are given below:

• classification

The pyspark.mllib library supports several classification methods such as binary classification, multiclass classification, and regression analysis. The object may belong to a different class. The objective of classification is to differentiate the data based on the information. Random Forest, Naive Bayes, Decision Tree are the most useful algorithms in classification.

• clustering

Clustering is an unsupervised machine learning problem. It is used when you do not know how to classify the data; we require the algorithm to find patterns and classify the data accordingly. The popular clustering algorithms are the K-means clustering, Gaussian mixture model, Hierarchical clustering.

• fpm

The fpm means frequent pattern matching, which is used for mining various items, itemsets, subsequences, or other substructure. It is mostly used in large-scale datasets.

• linalg

The mllib.linalg utilities are used for linear algebra.

• recommendation

It is used to define the relevant data for making a recommendation. It is capable of predicting future preference and recommending the top items. For example, Online entertainment platform Netflix has a huge collection of movies, and sometimes people face difficulty in selecting the favorite items. This is the field where the recommendation plays an important role.

• mllib regression

The regression is used to find the relationship and dependencies between variables. It finds the correlation between each feature of data and predicts the future values.

The mllib package supports many other algorithms, classes, and functions. Here we will understand the basic concept of pyspak.mllib.

MLlib Features

The PySpark mllib is useful for iterative algorithms. The features are the following:

• Extraction: It extracts features from “row” data.
• Transformation: It is used for scaling, converting, or modifying features.
• Selection: Selecting a useful subset from a larger set of features.
• Locality Sensitive Hashing: It combines aspects of feature transformation with other algorithms.

Let’s have a look at the essential libraries of PySpark MLlib.

MLlib Linear Regression

Linear regression is used to find the relationship and dependencies between variables. Consider the following code:

frompyspark.sql import SparkSession  

spark = SparkSession.builder.appName('Customer').getOrCreate()  

frompyspark.ml.regression import LinearRegression  

dataset = spark.read.csv(r'C:\Users\DEVANSH SHARMA\Ecommerce-Customers.csv')  

dataset.show(10)

Output:+——————–+——————–+—————-+——————+——————+——————+——————–+——————-+ | _c0| _c1| _c2| _c3| _c4| _c5| _c6| _c7| +——————–+——————–+—————-+——————+——————+——————+——————–+——————-+ | Email| Address| Avatar|Avg Session Length| Time on App| Time on Website|Length of Membership|Yearly Amount Spent| |mstephenson@ferna…|835 Frank TunnelW…| Violet| 34.49726772511229| 12.65565114916675| 39.57766801952616| 4.0826206329529615| 587.9510539684005| | [email protected]|4547 Archer Commo…| DarkGreen| 31.92627202636016|11.109460728682564|37.268958868297744| 2.66403418213262| 392.2049334443264| | [email protected]|24645 Valerie Uni…| Bisque|33.000914755642675|11.330278057777512|37.110597442120856| 4.104543202376424| 487.54750486747207| |riverarebecca@gma…|1414 David Throug…| SaddleBrown| 34.30555662975554|13.717513665142507| 36.72128267790313| 3.120178782748092| 581.8523440352177| |mstephens@davidso…|14023 Rodriguez P…|MediumAquaMarine| 33.33067252364639|12.795188551078114| 37.53665330059473| 4.446308318351434| 599.4060920457634| |alvareznancy@luca…|645 Martha Park A…| FloralWhite|33.871037879341976|12.026925339755056| 34.47687762925054| 5.493507201364199| 637.102447915074| |katherine20@yahoo…|68388 Reyes Light…| DarkSlateBlue| 32.02159550138701|11.366348309710526| 36.68377615286961| 4.685017246570912| 521.5721747578274| | [email protected]|Unit 6538 Box 898…| Aqua|32.739142938380326| 12.35195897300293| 37.37335885854755| 4.4342734348999375| 549.9041461052942| |vchurch@walter-ma…|860 Lee KeyWest D…| Salmon| 33.98777289568564|13.386235275676436|37.534497341555735| 3.2734335777477144| 570.2004089636196| +——————–+——————–+—————-+——————+——————+——————+——————–+——————-+ only showing top 10 rows

In the following code, we are importing the VectorAssembler library to create a new column Independent feature:

frompyspark.ml.linalg import Vectors  

frompyspark.ml.feature import VectorAssembler  

featureassembler = VectorAssembler(inputCols = ["Avg Session Length","Time on App","Time on Website"],outputCol = "Independent Features")  

output = featureassembler.transform(dataset)  

output.show()

Output:+——————+ Independent Feature +——————+ |34.49726772511229 | |31.92627202636016 | |33.000914755642675| |34.30555662975554 | |33.33067252364639 | |33.871037879341976| |32.02159550138701 | |32.739142938380326| |33.98777289568564 | +——————+

z = featureassembler.transform(dataset)  

finlized_data = z.select("Indepenent feature", "Yearly Amount Spent",)  

z.show()

Output:+——————–++——————-+ |Independent Feature | Yearly Amount Spent| +——————–++——————-+ |34.49726772511229 | 587.9510539684005 | |31.92627202636016 | 392.2049334443264 | |33.000914755642675 | 487.5475048674720 | |34.30555662975554 | 581.8523440352177 | |33.33067252364639 | 599.4060920457634 | |33.871037879341976 | 637.102447915074 | |32.02159550138701 | 521.5721747578274 | |32.739142938380326 | 549.9041461052942 | |33.98777289568564 | 570.2004089636196 | +——————–++——————-+

PySpark provides the LinearRegression() function to find the prediction of any given dataset. The syntax is given below:

regressor = LinearRegression(featureCol = 'column_name1', labelCol = 'column_name2 ')  

MLlib K- Mean Cluster

The K- Mean cluster algorithm is one of the most popular and commonly used algorithms. It is used to cluster the data points into a predefined number of clusters. The below example is showing the use of MLlib K-Means Cluster library:

from pyspark.ml.clustering import KMeans  

from pyspark.ml.evaluation import ClusteringEvaluator  

# Loads data.  

dataset = spark.read.format("libsvm").load(r"C:\Users\DEVANSH SHARMA\Iris.csv")  

# Trains a k-means model.  

kmeans = KMeans().setK(2).setSeed(1)  

model = kmeans.fit(dataset)  

# Make predictions  

predictions = model.transform(dataset)  

# Evaluate clustering by computing Silhouette score  

evaluator = ClusteringEvaluator()  

silhouette = evaluator.evaluate(predictions)  

print("Silhouette with squared euclidean distance = " + str(silhouette))  

# Shows the result.  

centers = model.clusterCenters()  

print("Cluster Centers: ")  

for center in centers:  

    print(center)

Parameters of PySpark MLlib

The few important parameters of PySpark MLlib are given below:

• Ratings

It is RDD of Ratings or (userID, productID, rating) tuple.

• Rank

It represents Rank of the computed feature matrices (number of features).

• Iterations

It represents the number of iterations of ALS. (default: 5)

• Lambda

It is the Regularization parameter. (default : 0.01)

• Blocks

It is used to parallelize the computation of some number of blocks.

Collaborative Filtering (mllib.recommendation)

Collaborative filtering is a technique that is generally used for a recommender system. This technique is focused on filling the missing entries of a user-item. Association matrix spark.ml currently supports model-based collaborative filtering. In collaborative filtering, users and products are described by a small set of hidden factors that can be used to predict missing entries.

Scaling of the regularization parameter

The regularization parameter regParam is scaled to solve least-squares problem. The least-square problem occurs when the number of ratings are user-generated in updating user factors, or the number of ratings the product received in updating product factors.

Cold-start strategy

The ALS Model (Alternative Least Square Model) is used for prediction while making a common prediction problem. The problem encountered when user or items in the test dataset occurred that may not be present during training the model. It can occur in the two scenarios which are given below:

• In the prediction, the model is not trained for users and items that have no rating history (it is called a cold-start strategy).
• The data is splitted between training and evaluation sets during cross-validation. It is widespread to encounter users and items in the evaluation set that are not in the training set.

Let’s consider the following example, where we load ratings data from the MovieLens dataset. Each row is containing a user, a movie, rating and a timestamp.

#importing the libraries  

frompyspark.ml.evaluation import RegressionEvaluator  

frompyspark.ml.recommendation import ALS  

frompyspark.sql import Row  

no_of_lines = spark.read.text(r"C:\Users\DEVANSH SHARMA\MovieLens.csv").rdd  

no_of_parts = no_of_lines.map(lambda row: row.value.split("::"))  

ratingsRDD = no_of_lines.map(lambda p: Row(userId=int(p[0]), movieId=int(p[1]),  

                                     rating=float(p[2]), timestamp=long(p[3])))  

ratings = spark.createDataFrame(ratingsRDD)  

(training, test) = ratings.randomSplit([0.8, 0.2])  

  

# Develop the recommendation model using ALS on the training data  

# Note we set cold start strategy to make sure that we don't get NaN evaluation metrics.  

als = ALS(maxIter=5, regParam=0.01, userCol="userId", itemCol="movieId", ratingCol="rating",  

    coldStartStrategy="drop")  

model = als.fit(training)  

  

# Calculate the model by computing the RMSE on the test data  

predictions = model.transform(test)  

evaluator = RegressionEvaluator(metricName="rmse", labelCol="rating",  

predictionCol="prediction")  

rmse = evaluator.evaluate(predictions)  

print("Root-mean-square error = " + str(rmse))  

  

# Evaluate top 10 movie recommendations for each user  

userRecs = model.recommendForAllUsers(10)  

# Evaluate top 10 user recommendations for each movie  

movieRecs = model.recommendForAllItems(10)  

# Evaluate top 10 movie recommendations for a specified set of users  

users = ratings.select(als.getUserCol()).distinct().limit(3)  

userSubsetRecs = model.recommendForUserSubset(users, 10)  

# Evalute top 10 user recommendations for a specified set of movies  

movies = ratings.select(als.getItemCol()).distinct().limit(3)  

movieSubSetRecs = model.recommendForItemSubset(movies, 10)

Data structure organizes the storage in computers so that we can easily access and change data. Stacks and Queues are the earliest data structure defined in computer science. A simple Python list can act as a queue and stack as well. A queue follows FIFO rule (First In First Out) and used in programming for sorting. It is common for stacks and queues to be implemented with an array or linked list.

Stack

A Stack is a data structure that follows the LIFO(Last In First Out) principle. To implement a stack, we need two simple operations:

• push – It adds an element to the top of the stack.
• pop – It removes an element from the top of the stack.

Operations:

• Adding – It adds the items in the stack and increases the stack size. The addition takes place at the top of the stack.
• Deletion – It consists of two conditions, first, if no element is present in the stack, then underflow occurs in the stack, and second, if a stack contains some elements, then the topmost element gets removed. It reduces the stack size.
• Traversing – It involves visiting each element of the stack.

Characteristics:

• Insertion order of the stack is preserved.
• Useful for parsing the operations.
• Duplicacy is allowed.

Code

# Code to demonstrate Implementation of   

# stack using list   

x = ["Python", "C", "Android"]   

x.push("Java")   

x.push("C++")   

print(x)   

print(x.pop())   

print(x)   

print(x.pop())   

print(x)

Output:[‘Python’, ‘C’, ‘Android’, ‘Java’, ‘C++’] C++ [‘Python’, ‘C’, ‘Android’, ‘Java’] Java [‘Python’, ‘C’, ‘Android’]

Queue

A Queue follows the First-in-First-Out (FIFO) principle. It is opened from both the ends hence we can easily add elements to the back and can remove elements from the front.

To implement a queue, we need two simple operations:

• enqueue – It adds an element to the end of the queue.
• dequeue – It removes the element from the beginning of the queue.

Operations on Queue

• Addition – It adds the element in a queue and takes place at the rear end, i.e., at the back of the queue.
• Deletion – It consists of two conditions – If no element is present in the queue, Underflow occurs in the queue, or if a stack contains some elements then element present at the front gets deleted.
• Traversing – It involves to visit each element of the queue.

Characteristics

• Insertion order of the queue is preserved.
• Duplicacy is allowed.
• Useful for parsing CPU task operations.

Note: The implementation of a queue is a little bit different. A queue follows the “First-In-First-Out”. Time plays an important factor here. The Stack is fast because we insert and pop the elements from the end of the list, whereas in the queue, the insertion and pops are made from the beginning of the list, so it becomes slow. The cause of this time difference is due to the properties of the list, which is fast in the end operation but slow at the beginning operations because all other elements have to be shifted one by one.

Code

import queue   

# Queue is created as an object 'L'  

L = queue.Queue(maxsize=10)   

  

# Data is inserted in 'L' at the end using put()   

L.put(9)   

L.put(6)   

L.put(7)   

L.put(4)   

# get() takes data from   

# from the head    

# of the Queue   

print(L.get())   

print(L.get())   

print(L.get())   

print(L.get())

Output:9 6 7 4

To add “magic” to the class we create, we can define special methods called “magic methods.” For example, the magic methods __init__ and __str__are always wrapped by double underscores from both sides. By granting us accessibility to Python’s built-in syntax tools, magic methods can improve the structure of our classes.

We can integrate Python’s built-in classes with our classes. The class which has inherited from the built-in class is known as a child class. A child class has access to all of the attributes of the parent class, including its methods. By utilizing the essential built-in features, we can customize some of the tasks of our class by using magic methods.

__init__ Method

After we have constructed an instance of the class, but before that instance is returned to the caller of the class, the _init_ method is executed. When we create an instance of the class, it is called automatically, just like constructors in various programming languages like the popular ones C++, Java, C#, PHP, etc. These methods are invoked after _new_ and therefore are referred to as initialising. We should define the instance parameters here.

Code

# Python program to show how __init__ method works  

  

# Creating a class  

class methods():  

    def __init__(self, *args):  

        print ("Now called __init__ magic method, after tha initialised parameters")  

        self.name = args[0]  

        self.std = args[1]  

        self.marks = args[2]  

  

Student = methods("Itika", 11, 98)  

print(Student)  

print(f"Name, standard, and marks of the student is: \n", Student.name, "\n", Student.std, "\n", Student.marks)

Output:Now called __init__ magic method, after tha initialised parameters <__main__.methods object at 0x3701290> Name, standard, and marks of the student is: Itika 11 98

__new__() Method

The magic method __new__() is called implicitly by the __init__() method. The new instance returned by the __new__() method is initialised. To modify the creation of objects in a user-defined class, we must supply a modified implementation of the __new__() magic method. We need to provide the first argument as the reference to the class whose object is to be created for this static function.

Code

# Python program to show how __new__ method works  

    

# Creating a class  

class Method(object):  

    def __new__( cls ):  

        print( "Creating an instance by __new__ method")  

        return super(Method, cls).__new__(cls)  

    # Calling the init method  

    def __init__( self ):  

        print( "Init method is called here" )  

  

Method()

Output:Creating an instance by __new__ method Init method is called here <__main__.Method at 0x30dfb88>

__add__ Method

We use the magic method __add__to add the class instance’s attributes. Consider the scenario where object1 belongs to class Method and object2 belongs to class Method 1, both of which have the same attribute called “attribute” that stores any value passed to the class while creating the instance. If specified to add the attributes, the __add__ function implicitly adds the instances’ same attributes, such as object1.attribute + object2.attribute, when the action object1 + object2 is completed.

Below is the code to show how we add two attributes of two instances of different classes without using the __add__ magic method.

Code

# Python program to show how to add two attributes  

  

# Creating a class  

class Method:  

    def __init__(self, argument):  

        self.attribute = argument  

  

# Creating a second class  

class Method_2:  

    def __init__(self, argument):  

        self.attribute = argument  

# creating the instances  

instance_1 = Method(" Attribute")  

print(instance_1.attribute)  

instance_2 = Method_2(" 27")  

print(instance_2.attribute)  

  

# Adding two attributes of the instances  

print(instance_2.attribute + instance_1.attribute)

Output:Attribute 27 27 Attribute By using __add__ magic method the code changes to this.

Code

# Python program to show how __add__ method works  

  

# Creating a class  

class Method:  

    def __init__(self, argument):  

        self.attribute = argument  

    def __add__(self, object1):  

        return self.attribute + object1.attribute  

  

# Creating a second class  

class Method_2:  

    def __init__(self, argument):  

        self.attribute = argument  

    def __add__(self, object1):  

        return self.attribute + object1.attribute  

instance_1 = Method(" Attribute")  

print(instance_1)  

instance_2 = Method_2(" 27")  

print(instance_2)  

print(instance_2 + instance_1)

Output:<__main__.Method object at 0x37470f0> <__main__.Method_2 object at 0x376beb8> 27 Attribute

Classes, Method and Method_1 in the script above have a property called “attribute” that stores a string. We create two instances, instance_1 and instances_2, with corresponding attributes of ” Attribute” and ” 27″ values. The __add__ method is used to translate the action instance_1 + instance_2 into instance_1 + instance_2.attribute, which produces output ( 27 Attribute).

__repr__ Method

The class instance is represented as a string using the magic method __repr__. The __repr__ method, which produces a string in the output, is automatically called whenever we attempt to print an object of that class.

Code

# Python program to show how __repr__ magic method works  

  

# Creating a class  

class Method:  

    # Calling __init__ method and initializing the attributes of the class  

    def __init__(self, x, y, z):  

        self.x = x  

        self.y = y  

        self.z = z  

    # Calling the __repr__ method and providing the string to be printed each time instance is printe  

    def __repr__(self):  

        return f"Following are the values of the attributes of the class Method:\nx = {self.x}\ny = {self.y}\nz = {self.z}"  

instance = Method(4, 6, 2)  

print(instance)

Output:Following are the values of the attributes of the class Method: x = 4 y = 6 z = 2

__contains__ Method

The ‘in’ membership operator of Python implicitly calls the __contains__ method. We can use the __contains__ method to determine if an element is contained in an object’s attributes. We can use this method for attributes that are containers ( such as lists, tuples, etc.).

Code

# Python code to show how the __contains__ magic method works  

  

# Creating a class  

class Method:  

    # Calling the __init__ method and initializing the attributes  

    def __init__(self, attribute):  

        self.attribute = attribute  

          

    # Calling the __contains__ method  

    def __contains__(self, attribute):  

        return attribute in self.attribute  

# Creating an instance of the class  

instance = Method([4, 6, 8, 9, 1, 6])  

  

# Checking if a value is present in the container attribute  

print("4 is contained in ""attribute"": ", 4 in instance)  

print("5 is contained in ""attribute"": ", 5 in instance)

Output:4 is contained in attribute: True 5 is contained in attribute: False

We have used the __contanis__ magic method in the program above to determine if a given integer is contained in the attribute “attribute”. In this case, “attribute” is a list of integers. The integer 4 is present in the given list of integers passed to the class Method as an attribute. While 5 is not present in the list.

__call__ Method

When a class instance is called, the Python interpreter calls the magic method __call__. We can utilise the __call__ method to explicitly call an operation using the instance name rather than creating an additional method to carry out specific activities.

Code

# Python program to show how the __call__ magic method works  

  

# Creating a class  

class Method:  

    # Calling the __init__ method and initializing the attributes  

    def __init__(self, a):  

        self.a = a  

    # Calling the __call__ method to multiply a number to the attribute value  

    def __call__(self, number):  

        return self.a * number  

  

# Creating an instance and proving the value to the attribute a  

instance = Method(7)  

print(instance.a) # Printing the value of the attribute a  

# Calling the instance while passing a value which will call the __call__ method  

print(instance(5))

Output:7 35

__iter__ Method

For the given instance, a generator object is supplied using the __iter__ method. To benefit from the __iter__ method, we can leverage the iter() and next() methods.

Code

# Python program to show how the __iter__ method works  

  

# Creating a class  

class Method:  

    def __init__(self, start_value, stop_value):  

        self.start = start_value  

        self.stop = stop_value  

    def __iter__(self):  

        for num in range(self.start, self.stop + 1):  

            yield num ** 2  

# Creating an instance  

instance = iter(Method(3, 8))  

print( next(instance) )  

print( next(instance) )  

print( next(instance) )  

print( next(instance) )  

print( next(instance) )  

print( next(instance) )

Output:9 16 25 36 49 64

We have calculated the squares of the numbers in the code. For the numbers in the specified range, squares are calculated in the program above (start and stop). The __iter__ method, which generates squares of the numbers between the given range, is called when we call the function iter(Method(3, 8)). In our example, we’re using the range of 3 to 8; therefore, calling the next() method will produce the results 9, 16, 25, 36, 49, 64.

The Python supports the programs that can be run on the command line, complete with command line arguments. It is the input parameter that needs to be passed to the script when executing them. It means to interact with a command-line interface for the scripts. There are different ways we can use command line arguments, few of them are:

1. Python sys module

It is a basic module that comes with Python distribution from the early versions. It is a similar approach as C library using argc/argv to access the arguments. The sys module implements command-line arguments in a simple list structure named sys.argv.

Each list element represents a single argument. The first one — sys.argv[0] — is the name of Python script. The other list elements are sys.argv[1] to sys.argv[n]- are the command line arguments 2 to n. As a delimiter between arguments, space is used. Argument values that contain space in it have to be quoted, accordingly.

It stores command-line arguments into a list; we can access it using sys.argv. This is very useful and a simple way to read command-line arguments as String.

Code

import sys  

# Check the type of sys.argv  

print(type(sys.argv))  #  <class ' list '>  

# Print the command line arguments  

print(' The command line arguments are: ')  

# Iterate over sys.argv and print each argument  

for i in sys.argv:  

    print(i)

Output:<class ‘ list ‘> The command line arguments are: script.py arg1 arg2

2. Python getopt module

The Python getopt module extends the separation of the input string by parameter validation. Based on getopt C function, it allows both short and long options, including a value assignment.

It is very similar to C getopt() function for parsing command line parameters.

It is useful in parsing command line arguments where we want the user to enter some options.

Code

import getopt  

import sys  

argv = sys.argv[1:]  

try:  

    opts, args = getopt.getopt(argv, 'hm:d', ['help', 'my_file='])  

    print(opts)  

    print(args)  

except getopt.GetoptError:  

    # Print a message or do something useful  

    print('Something went wrong!')  

    sys.exit(2)

Output:[(‘-h’, ”), (‘-m’, ‘my_value’), (‘–my_file’, ‘input.txt’)] [‘arg1’, ‘arg2’]

3. Python argparse module

It offers a command-line interface with standardized output, whereas the former two solutions leave most of the work in your hands. argparse allows verification of fixed and optional arguments with a name checking as either UNIX or GNU style. It is the preferred way to parse command-line arguments. It provides a lot of option such as positional arguments, the default value for arguments, helps message, specifying the data type of argument etc.

It makes it easy to write the user-friendly command-line interfaces. It automatically generates help and usage messages and issues errors when a user gives invalid arguments to the program. It means to communicate between the writer of a program and user which does not require going into the code and making changes to the script. It provides the ability to a user to enter into the command-line arguments.

Code

import argparse  

  

# Create an ArgumentParser object  

parser = argparse.ArgumentParser(description = 'Example script using argparse')  

  

# Add arguments  

parser.add_argument('-f',  '--file', help = 'Specify a file name')  

parser.add_argument('-v', '--verbose', action = 'store_true', help = 'Enable verbose mode')  

  

# Parse the command line arguments  

args = parser.parse_args()  

  

# Access the argument values  

if args.file:  

    print(f "File name : {args.file} ")  

if args.verbose:  

    print("Verbose mode is enabled")

Output:python script.py -f myfile.txt -v File name : myfile.txt Verbose mode is enabled

The above three are the common basic modules to operate command line arguments in Python. Other simple modules in Python for command line arguments are:

Docopt

Docopt is used to create command line interfaces. It simplifies the process of parsing command-line arguments and generating help messages. To use docopt, you need to install the library first. You can install it using pip:

Code

from docopt import docopt  

  

__doc__  =  """Usage: 

    my_program.py [--option1] [--option2=<value>] <argument> 

 

Options: 

    -h, --help         Show this help message. 

    -o, --option1      Enable option 1. 

    -t, --option2 = <value>  Specify option 2 value. 

"""  

  

if __name__ == '__main__':  

    arguments = docopt(__doc__, version = 'Example 1')  

    print(arguments)

Output:$ python script.py –option1 –option2=value argument_value { ‘–help’: False, ‘–option1’: True, ‘–option2’: ‘value’, ‘ ‘: ‘ argument_value ‘, ‘–version’: False }

Fire

Python Fire automatically generates a command line interface; you only need one line of code. Unlike the other modules, it works instantly. You don’t need to define any arguments; all the methods are linked by default. To install it, type:

pip install fire  

Define or use a class:

Code

import fire    

class Python(object):    

    def hello(self):    

    print("Hello")    

       def openfile(self, filename):    

        print(" Open file  '" + filename + "'")    

    

if __name__ == '__main__':    

    fire.Fire(Python)

Output:$ python script.py hello Hello $ python script.py openfile my_file.txt Open file ‘my_file.txt’

Command Line arguments Modules

Module	Use	Python version
sys	All arguments in sys.argv (basic)	All
argparse	Build a command line interface	>= 2.3
docopt	Created command line interfaces	>= 2.5
fire	Automatically generate command line interfaces (CLIs)	All
optparse	Deprecated	< 2.7

Introduction:

In this article, we are discussing Arrays in Python. The Array is used in every programming language, like C, C++, Java, Python, R, JavaScript, etc. By using an array, we can store more than one data. The Array is a process of memory allocation. It is performed as a dynamic memory allocation. We can declare an array like x[100], storing 100 data in x. It is a container that can hold a fixed number of items, and these items should be the same type. An array is popular in most programming languages like C/C++, JavaScript, etc.

The Array is an idea of storing multiple items of the same type together, making it easier to calculate the position of each element by simply adding an offset to the base value. A combination of the arrays could save a lot of time by reducing the overall size of the code. It is used to store multiple values in a single variable. If you have a list of items that are stored in their corresponding variables like this:

car1 = "Lamborghini"  

car2 = "Bugatti"  

car3 = "Koenigsegg"

If you want to loop through cars and find a specific one, you can use the Array. You can use an array to store more than one item in a specific variable.

The Array can be handled in Python by a module named Array. It is useful when we must manipulate only specific data values. The following are the terms to understand the concept of an array:

Element – Each item stored in an array is called an element.

Index – The location of an element in an array has a numerical index, which is used to identify the element’s position. The index value is very much important in an Array.

Array Representation:

An array can be declared in various ways and in different languages. The important points that should be considered are as follows:

1. The index starts with 0.
2. We can easily find any elements within this Array using the Index value.
3. The length of the Array defines the capacity to store the elements. It is written like x[100], which means the length of array x is specified by 100.

Array operations

Some of the basic operations supported by an array are as follows:

• Traverse – It prints all the elements one by one.
• Insertion – It adds an element at the given index.
• Deletion – It deletes an element at the given index.
• Search – It searches an element using the given index or by the value.
• Update – It updates an element at the given index.

The Array can be created in Python by importing the array module to the python program.

from array import *  

arrayName = array(typecode, [initializers])

Accessing array elements

We can access the array elements using the respective indices of those elements.

Program code:

Here we give an example of how we access the elements of an array using its index value in Python. The code is given below –

import array as arr    

a = arr.array('i', [2, 4, 5, 6])    

print("First element is:", a[0])    

print("Second element is:", a[1])   

print("Third element is:", a[2])  

print("Forth element is:", a[3])  

print("last element is:", a[-1])    

print("Second last element is:", a[-2])    

print("Third last element is:", a[-3])    

print("Forth last element is:", a[-4])    

print(a[0], a[1], a[2], a[3], a[-1],a[-2],a[-3],a[-4])

Output:

Now we compile the above code in python, and after successful compilation, we run it. Then the output is given below -First element is: 2 Second element is: 4 Third element is: 5 Forth element is: 6 last element is: 6 Second last element is: 5 Third last element is: 4 Forth last element is: 2 2 4 5 6 6 5 4 2

Explanation:

In the above example, we have imported an array, defined a variable named “a” that holds the elements of an array, and printed the elements by accessing elements through the indices of an array. Here we can easily find out the array element by using the array index like a[0], a[1], a[-1], and so on.

How to change or add elements?

Arrays are mutable, and their elements can be changed similarly to lists.

Program code:

Here in this example, we can change or add or replace any element from the Array in Python. The code is given below –

import array as arr    

numbers = arr.array('i', [1, 2, 3, 5, 7, 10])    

     

# changing first element 1 by the value 0.  

numbers[0] = 0       

print(numbers)          # Output: array('i', [0, 2, 3, 5, 7, 10])  

  

# changing last element 10 by the value 8.  

numbers[5] = 8      

print(numbers)          # Output: array('i', [0, 2, 3, 5, 7, 10])      

     

# replace the value of 3rd to 5th element by 4, 6 and 8  

numbers[2:5] = arr.array('i', [4, 6, 8])      

print(numbers)          # Output: array('i', [0, 2, 4, 6, 8, 10])

Output:

Now we compile the above code in python, and after successful compilation, we run it. Then the output is given below -array(‘i’, [0, 2, 3, 5, 7, 10]) array(‘i’, [0, 2, 3, 5, 7, 8]) array(‘i’, [0, 2, 4, 6, 8, 8])

Explanation:

In the above example, we have imported an array and defined a variable named “numbers,” which holds the value of an array. If we want to change or add the elements in an array, we can do it by defining the index of an array where we want to change or add the elements. Here we just mentioned the index number of elements you want to change and declared the new value by which you want to replace the old elements.

Why use Arrays in Python?

A combination of arrays saves a lot of time. The Array can reduce the overall size of the code. Using an array, we can solve a problem quickly in any language. The Array is used for dynamic memory allocation.

How to Delete Elements from an Array?

The elements can be deleted from an array using Python’s del statement. If we want to delete any value from the Array, we can use the indices of a particular element.

import array as arr  

number = arr.array('i', [1, 2, 3, 3, 4])  

del number[2]                           # removing third element  

print(number)                           # Output: array('i', [1, 2, 3, 4])

Output:

Now we compile the above code in python, and after successful compilation, we run it. Then the output is given below -array(‘i’, [10, 20, 40, 60])

Explanation: In the above example, we have imported an array and defined a variable named as “number” which stores the values of an array. Here, by using del statement, we are removing the third element [3] of the given array.

Finding the length of an array

The length of an array is defined as the number of elements present in an array. It returns an integer value that is equal to the total number of the elements present in that array.

Syntax

len(array_name)

Array Concatenation

We can easily concatenate any two arrays using the + symbol.

Example 1:

a=arr.array('d',[1.1 , 2.1 ,3.1,2.6,7.8])  

b=arr.array('d',[3.7,8.6])  

c=arr.array('d')  

c=a+b  

print("Array c = ",c)

Output:

Now we compile the above code in python, and after successful compilation, we run it. Then the output is given below -Array c= array(‘d’, [1.1, 2.1, 3.1, 2.6, 7.8, 3.7, 8.6])

Explanation

In the above example, we have defined variables named as “a, b, c” that hold the values of an array.

import array as arr    

x = arr.array('i', [4, 7, 19, 22])  # Initialize the array elements  

print("First element:", x[0])    

print("Second element:", x[1])    

print("Second last element:", x[-1])

Output:First element: 4 Second element: 7 Second last element: 22

Now we compile the above code in python, and after successful compilation, we run it. Then the output is given below –

Explanation:

In the above example, first, we imported an array and defined a variable named “x,” which holds the value of an array. Then, we printed the elements of an array using the index value of this array element.

The term “IDE” refers for “Integrated Development Environment,” which is a coding tool that aids in automating the editing, compiling, testing, and other steps of an SDLC while making it simple for developers to execute, write, and debug code.

It is specifically made for software development and includes a number of tools that are used in the creation and testing of the software.

There are some Python IDEs which are as follows:

• PyCharm
• Spyder
• PyDev
• Atom
• Wing
• Jupyter Notebook
• Thonny
• Rodeo
• Microsoft Visual Studio
• Eric

---

PyCharm

The Jet Brains created PyCharm, a cross-platform Integrated Development Environment (IDE) created specifically for Python. It is the most popular IDE and is accessible in both a premium and a free open-source version. By handling everyday duties, a lot of time is saved.

It is a full-featured Python IDE with a wealth of features including auto code completion, easy project navigation, quick error checking and correction, support for remote development, database accessibility, etc.

Features

• Smart code navigation
• Errors Highlighting
• Powerful debugger
• Supports Python web development frameworks, i.e., Angular JS, Javascript

---

Spyder

Spyder is a well-known open-source IDE that is best suited for data research and has a high level of recognition in the industry. Scientific Python Development Environment is Spyder’s full name. It supports all popular operating systems, including Windows, MacOS X, and Linux.

A number of features are offered by it, including a localised code editor, a document viewer, a variable explorer, an integrated console, etc. It also supports a number of scientific modules, including SciPy and NumPy.

Features

• Proper syntax highlighting and auto code completion
• Integrates strongly with IPython console
• Performs well in multi-language editor and auto code completion mode

---

PyDev

As an external plugin for Eclipse, PyDev is one of the most popular Python IDEs. The Python programmers who have a background in Java naturally gravitate towards this Python interpreter because it is so well-liked by users.

In 2003-2004, Aleksandar Totic, who is well known for his work on the Mosaic browser, contributed to the Pydev project.

Django integration, code auto-completion, smart and block indents, among other features, are features of Pydev.

Features

• Strong Parameters like refactoring, debugging, code analysis, and code coverage function.
• It supports virtual environments, Mypy, and black formatter.
• Also supports PyLint integration, remote debugger, Unit test integration, etc.

---

Atom

GitHub, a company that was first founded as an open-source, cross-platform project, is the company that creates Atom. It is built on the Electron framework, which enables cross-platform desktop applications utilising Chromium and Node.js and is dubbed the “Hackable Text Editor for the 21st Century.”

Features

• Visualize the results on Atom without open any other window.
• A plugin named “Markdown Preview Plus” provides built-in support for editing and visualizing Markdown files.

---

Wing

It’s described as a cross-platform IDE with a tonne of useful features and respectable development support. It is free to use in its personal edition. The 30-day trial period for the pro version is provided for the benefit of the developers.

Features

• Customizable and can have extensions as well.
• Supports remote development, test-driven development along with the unit test.

---

Jupyter Notebook

Jupyter is one of the most used Python notebook editors that is used across the Data Science industry. You can create and edit notebook documents using this web application, which is based on the server-client architecture. It utilises Python’s interpretive nature to its fullest potential.

Features

• Supports markdowns
• Easy creation and editing of codes
• Ideal for beginners in data science

---

Thonny

Thonny is a Python IDE (Integrated Development Environment) that is open-source, free, and geared towards beginners. Since its initial release in 2016, it has grown to be a well-liked option for novice Python coders.

Thonny’s user-friendly interface is one of its most distinguishing qualities. It makes it simple for beginners to learn Python and debug their programmes because it incorporates a code editor, debugger, and REPL (Read-Eval-Print-Loop) in a single window. To assist users with writing proper code, Thonny also has tools like code completion, syntax highlighting, and error highlighting.

Thonny IDE that works well for teaching and learning programming is Thonny. Software that highlights syntax problems and aids code completion was created at the University of Tartu.

Features

• Simple debugger
• Supports highlighting errors and auto code completion

---

Rodeo

When it comes to gathering data and information from many sources for data science projects, Rodeo is considered one of the top Python IDEs.

It offers code auto-completion and cross-platform capability.

Features

• Allows the functions for comparing data, interact, plot, and inspect data.
• Auto code completion, syntax highlighter, visual file navigator, etc.

---

Microsoft Visual Studio

Microsoft Visual Studio is an open-source code editor which was best suited for development and debugging of latest web and cloud projects. It has its own marketplace for extensions.

An integrated development environment (IDE) called Microsoft Visual Studio is used to create software for the Windows, Android, and iOS operating systems. Since its initial release in 1997, it has grown to be a well-liked software development tool.

Code editing, debugging, and code analysis are just a few of the capabilities and tools that are included in the IDE. It supports a variety of programming languages, including Python, C++, C#, Visual Basic, and others. Additionally, Microsoft Visual Studio comes with a variety of project templates that make it simpler for developers to get started on their projects right away.

Microsoft Visual Studio 2022, the most recent release, comes with new features like improved debugging and testing capabilities, improved Git integration, and a revamped user interface. The enhanced performance of the IDE makes it quicker and more effective to construct complicated software projects.

Features

• Supports Python Coding in Visual studio
• Available in both paid and free version

---

Eric Python

The Eric Python is a Python-based editor that may be used for both professional and non-professional tasks.

Since its initial release in 2000, Eric IDE (Integrated Development Environment) has been a free and open-source Python IDE. It offers programmers a setting in which to efficiently write, test, and debug Python programmes since it is user-friendly and simple to use.

Python 2 and 3 are among the Python versions that are supported by Eric IDE, which also offers features like code highlighting, code completion, and syntax checking. Additionally, it contains an integrated debugger that enables programmers to effectively debug their programmes.

The Eric IDE’s plugin system, which enables developers to increase its capabilities, is one of its primary features. An integrated version control system, a database browser, and a Python profiler are just a few of the plugins that are available for the Eric IDE.

The Python collection module is defined as a container that is used to store collections of data, for example – list, dict, set, and tuple, etc. It was introduced to improve the functionalities of the built-in collection containers.

Python collection module was first introduced in its 2.4 release.

There are different types of collection modules which are as follows:

namedtuple()

The Python namedtuple() function returns a tuple-like object with names for each position in the tuple. It was used to eliminate the problem of remembering the index of each field of a tuple object in ordinary tuples.

Examples

pranshu = ('James', 24, 'M')    

print(pranshu)

Output:(‘James’, 24, ‘M’)

OrderedDict()

The Python OrderedDict() is similar to a dictionary object where keys maintain the order of insertion. If we try to insert key again, the previous value will be overwritten for that key.

Example

import collections    

d1=collections.OrderedDict()    

d1['A']=10    

d1['C']=12    

d1['B']=11    

d1['D']=13    

    

for k,v in d1.items():    

    print (k,v)

Output:A 10 C 12 B 11 D 13

defaultdict()

The Python defaultdict() is defined as a dictionary-like object. It is a subclass of the built-in dict class. It provides all methods provided by dictionary but takes the first argument as a default data type.

Example

from collections import defaultdict      

number = defaultdict(int)      

number['one'] = 1      

number['two'] = 2      

print(number['three'])

Output:0

Counter()

The Python Counter is a subclass of dictionary object which helps to count hashable objects.

Example

from collections import Counter      

c = Counter()    

list = [1,2,3,4,5,7,8,5,9,6,10]      

Counter(list)    

Counter({1:5,2:4})      

list = [1,2,4,7,5,1,6,7,6,9,1]      

c = Counter(list)      

print(c[1])

Output:3

deque()

The Python deque() is a double-ended queue which allows us to add and remove elements from both the ends.

Example

from collections import deque    

list = ["x","y","z"]    

deq = deque(list)    

print(deq)

Output:deque([‘x’, ‘y’, ‘z’])

Chainmap Objects

A chainmap class is used to groups multiple dictionary together to create a single list. The linked dictionary stores in the list and it is public and can be accessed by the map attribute. Consider the following example.

Example

from collections import ChainMap  

baseline = {'Name': 'Peter', 'Age': '14'}  

adjustments = {'Age': '14', 'Roll_no': '0012'}  

print(list(ChainMap(adjustments, baseline)))

Output:[‘Name’, ‘Age’, ‘Roll_no’ ]

UserDict Objects

The UserDict behaves as a wrapper around the dictionary objects. The dictionary can be accessed as an attribute by using the UserDict object. It provides the easiness to work with the dictionary.

It provides the following attribute.

data – A real dictionary used to store the contents of the UserDict class.

UserList Objects

The UserList behaves as a wrapper class around the list-objects. It is useful when we want to add new functionality to the lists. It provides the easiness to work with the dictionary.

It provides the following attribute.

data – A real list is used to store the contents of the User class.

UserString Objects

The UserList behaves as a wrapper class around the list objects. The dictionary can be accessed as an attribute by using the UserString object. It provides the easiness to work with the dictionary.

It provides the following attribute.

data – A real str object is used to store the contents of the UserString class.

Introduction:

In this tutorial, we are discussing list comprehension in Python. Python is known for helping us produce code that is elegant, simple to write, and reads almost as well as plain English. List comprehension is one of the language’s most distinguishing features, allowing us to develop sophisticated functionality with just one line of code. On the other hand, many Python writers struggle to fully utilize the more complex aspects of list comprehension. Sometimes programmers may overuse them, resulting in much less efficient and difficult-to-read code.

Example:

Here we give the basic example of List comprehension in Python. The code is given below –

Person = ["Piyali", "Hiya", "Rudrashish", "Badsha", "Lipi"]  

newlist = [x for x in Person if "i" in x]  

print(newlist)

Output:

Now we compile the above code in Python, and after successful compilation, we run it. Then the output is given below -[‘Piyali’, ‘Hiya’, ‘Rudrashish’, ‘Lipi’]

Syntax:

The syntax of List comprehension in Python is given below –

newlist = [expression for item in iterable if condition == True]  

Here we are showing the basic use of list comprehension.

Program Code:

Now we give code examples without using list comprehension; how can we square every number of a list using just a for loop in Python? The code is given below –

#using for loop to iterate through items in list  

numbers = [3, 5, 1, 7, 3, 9]  

num = []  

  

for n in numbers:  

    num.append(n**2)  

  

print(num)

Output:

Now we compile the above code in Python, and after successful compilation, we run it. Then the output is given below -[9, 25, 1, 49, 9, 81]

This can be accomplished with only a single line of code using list comprehension.

Program Code:

Now we give code examples of list comprehension in Python for square every number of a list. The code is given below –

#using list comprehension to iterate through list items  

numbers = [3, 5, 1, 7, 3, 9]  

  

num = [n**2 for n in numbers]  

  

print(num)

Output:

Now we compile the above code in Python, and after successful compilation, we run it. Then the output is given below –

Benefits of Using List Comprehensions

There are so many advantages or benefits of using list comprehension in Python. The advantages are given below –

1. Loops and maps:

Loops and maps are typically regarded as more Pythonic than list comprehensions. But rather than taking that judgment at face value, it is worth considering the advantages of utilizing a list comprehension in Python over the alternatives. We will learn about a couple of cases when the alternatives are preferable options later.

2. The Single tool uses:

One of the essential advantages of utilizing list comprehension in Python is that it is a single tool that can be used in various circumstances. We don’t need to adopt a new strategy for each situation. List comprehensions may be leveraged for mapping or filtering and basic list generation.

3. Does not depend on appropriate parameter:

List comprehensions are regarded as Pythonic, as Python emphasizes simple, effective tools that can be used in many scenarios. As a bonus, we won’t have to remember the appropriate order of parameters when using a list comprehension in Python, as we would when calling map().

4. Easy to use:

List comprehensions are easier to read and grasp than loops since they are more declarative. We must concentrate on how exactly the List is constructed while using loops. We must manually build an empty list, loop over the List’s entries, and add each one to the List’s end. Instead, using a list comprehension in Python, we can concentrate on what we want to put in the List and allow Python to handle the list generation.

Program Code:

# Import module to keep track of time  

import time  

   

# defining function to execute for loop  

def for_loop(num):  

    l = []  

    for i in range(num):  

        l.append(i + 10)  

    return l  

   

# defining function to execute list comprehension  

def list_comprehension(num):  

    return [i + 10 for i in range(num)]  

   

# Giving values to the functions  

   

# Calculating time taken by for loop  

start = time.time()  

for_loop(10000000)  

end = time.time()  

  

print('Time taken by for loop:', (end - start))  

   

# Calculating time taken by list comprehension  

start = time.time()  

list_comprehension(10000000)  

end = time.time()  

  

print('Time taken by list comprehension:', (end - start))

Output:

Now we compile the above code in Python, and after successful compilation, we run it. Then the output is given below -Time taken by for loop: 7.005999803543091 Time taken by list comprehension: 2.822999954223633

Using List Comprehension to Iterate through String

List comprehension can also be used in the case of strings, as they are iterables.

Program Code:

Now we give code examples of using list comprehension in Python to iterate a string given in code. The code is given below –

letters = [ alpha for alpha in 'javatpoint' ]  

print( letters)

Output:[‘j’, ‘a’, ‘v’, ‘a’, ‘t’, ‘p’, ‘o’, ‘i’, ‘n’, ‘t’]

Using Conditions in List Comprehension

Conditional statements can be used by list comprehensions to change existing lists (or other tuples). We’ll make a list with mathematical operators, numbers, and a range of values.

Program Code:

number_list = [ num for num in range(30) if num % 2 != 0]  

print(number_list)

Output:

Now we compile the above code in Python, and after successful compilation, we run it. Then the output is given below -[1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29]

Addition of odd elements to the List:

Here we sum the odd elements from the given List. It is an example of list comprehension in Python. The code is given below –

def Sum(n):  

    dsum = 0  

    for ele in str(n):  

        dsum += int(ele)  

    return dsum  

List = [47, 69, 73, 97, 105, 845, 307]  

newList = [Sum(i) for i in List if i & 1]  

print(newList)

Output:

Now we compile the above code in Python, and after successful compilation, we run it. Then the output is given below -[11, 15, 10, 16, 6, 17, 10]

Nested List Comprehensions

Nested List Comprehensions are similar to nested for loops in that they are a list comprehension inside another list comprehension. The programme that implements nested loop is as follows:

Program Code:

nested_list = []  

   

for _ in range(3):  

       

    # Append an empty sublist inside the list  

    nested_list.append([])  

       

    for __ in range(5):  

        nested_list[_].append(__ + _)  

           

print(nested_list)

Output:

Now we compile the above code in Python, and after successful compilation, we run it. Then the output is given below -[[0, 1, 2, 3, 4], [1, 2, 3, 4, 5], [2, 3, 4, 5, 6]]

The same result may be created in fewer lines of code by utilizing layered list comprehensions.

Program Code:

The code for nested list comprehension is given below –

# Nested list comprehension  



nested_list = [[_ + __ for _ in range(5)] for __ in range(3)]  



   



print(nested_list) 

Output:

Now we compile the above code in Python, and after successful compilation, we run it. Then the output is given below -[[0, 1, 2, 3, 4], [1, 2, 3, 4, 5], [2, 3, 4, 5, 6]]

List comprehension is a powerful tool for describing and creating new lists based on existing ones. In general, list comprehension is lighter and easier to use than traditional list construction functions and loops. To provide user-friendly code, we should avoid writing large codes for list comprehensions. Every interpretation of the list or other iterables can be recast in a for loop, but not all the for loops can be rebuilt in the framework of list comprehension.

Python assert keyword is defined as a debugging tool that tests a condition. The Assertions are mainly the assumption that asserts or state a fact confidently in the program. For example, while writing a division function, the divisor should not be zero, and you assert that the divisor is not equal to zero.

It is merely a Boolean expression that has a condition or expression checks if the condition returns true or false. If it is true, the program does not do anything, and it moves to the next line of code. But if it is false, it raises an AssertionError exception with an optional error message.

The main task of assertions is to inform the developers about unrecoverable errors in the program like “file not found”, and it is right to say that assertions are internal self-checks for the program. It is the most essential for the testing or quality assurance in any application development area. The syntax of the assert keyword is given below.

Syntax

assert condition, error_message(optional)    

Why Assertion is used

It is a debugging tool, and its primary task is to check the condition. If it finds that the condition is true, it moves to the next line of code, and If not, then stops all its operations and throws an error. It points out the error in the code.

Where Assertion in Python used

• Checking the outputs of the functions.
• Used for testing the code.
• In checking the values of arguments.Checking the valid input.

Example1

This example shows the working of assert with the error message.

def avg(scores):    

    assert len(scores) != 0,"The List is empty."    

    return sum(scores)/len(scores)    

    

scores2 = [67,59,86,75,92]    

print("The Average of scores2:",avg(scores2))    

    

scores1 = []    

print("The Average of scores1:",avg(scores1))

Output:The Average of scores2: 75.8 AssertionError: The List is empty.

Explanation: In the above example, we have passed a non-empty list scores2 and an empty list scores1 to the avg() function. We received an output for scores2 list successfully, but after that, we got an error AssertionError: List is empty. The assert condition is satisfied by the scores2 list and lets the program continue to run. However, scores1 doesn’t satisfy the condition and gives an AssertionError.

Example2:

This example shows the “Divide by 0 error” in the console.

# initializing number     

x = 7    

y = 0    

# It uses assert to check for 0     

print ("x / y value is : ")     

assert y != 0, "Divide by 0 error"    

print (x / y)

Output:

x / y value is :

Runtime Exception :

Traceback (most recent call last): File “main.py”, line 6, in <module> assert y != 0, “Divide by 0 error” AssertionError: Divide by 0 error

Explanation:

In the above example, we have initialized an integer variable, i.e., x=7, y=0, and try to print the value of x/y as an output. The Python interpreter generated a Runtime Exception because of the assert keyword found the divisor as zero then displayed “Divide by 0 error” in the console.

Simple Mail Transfer Protocol (SMTP) is used as a protocol to handle the email transfer using Python. It is used to route emails between email servers. It is an application layer protocol which allows to users to send mail to another. The receiver retrieves email using the protocols POP(Post Office Protocol) and IMAP(Internet Message Access Protocol).

When the server listens for the TCP connection from a client, it initiates a connection on port 587.

Python provides a smtplib module, which defines an the SMTP client session object used to send emails to an internet machine. For this purpose, we have to import the smtplib module using the import statement.

$ import smtplib  

The SMTP object is used for the email transfer. The following syntax is used to create the smtplib object.

import smtplib     



smtpObj = smtplib.SMTP(host, port, local_hostname) 

It accepts the following parameters.

• host: It is the hostname of the machine which is running your SMTP server. Here, we can specify the IP address of the server like (https://www. smartstart.com) or localhost. It is an optional parameter.
• port: It is the port number on which the host machine is listening to the SMTP connections. It is 25 by default.
• local_hostname: If the SMTP server is running on your local machine, we can mention the hostname of the local machine.

The sendmail() method of the SMTP object is used to send the mail to the desired machine. The syntax is given below.

1. smtpObj.sendmail(sender, receiver, message)    

Example

#!/usr/bin/python3    

import smtplib    

sender_mail = '[email protected]'    

receivers_mail = ['[email protected]']    

message = """From: From Person %s  

To: To Person %s  

Subject: Sending SMTP e-mail   

This is a test e-mail message.  

"""%(sender_mail,receivers_mail)    

try:    

   smtpObj = smtplib.SMTP('localhost')    

   smtpObj.sendmail(sender_mail, receivers_mail, message)    

   print("Successfully sent email")    

except Exception:    

   print("Error: unable to send email")

Sending email from gmail

There are cases where the emails are sent using the Gmail SMTP server. In this case, we can pass Gmail as the SMTP server instead of using the localhost with the port 587.

Use the following syntax.

1. $ smtpObj = smtplib.SMTP(“gmail.com”, 587)     

Here, we need to login to the Gmail account using Gmail user name and password. For this purpose, the smtplib provide the login() method, which accepts the username and password of the sender.

This may make your Gmail ask you for access to less secure apps if you’re using Gmail. You will need to turn this ON temporarily for this to work.

Consider the following example.

Example

#!/usr/bin/python3    



import smtplib    



sender_mail = '[email protected]'    



receivers_mail = ['[email protected]']    



message = """From: From Person %s  



To: To Person %s  



Subject: Sending SMTP e-mail   



This is a test e-mail message.  



"""%(sender_mail,receivers_mail)    



try:    



   password = input('Enter the password');    



   smtpObj = smtplib.SMTP('gmail.com',587)    



   smtpobj.login(sender_mail,password)    



   smtpObj.sendmail(sender_mail, receivers_mail, message)    



   print("Successfully sent email")    



except Exception:    



   print("Error: unable to send email")

Sending HTML in email

We can format the HTML in the message by specifying the MIME version, content-type, and character set to send the HTML.

Consider the following example.

Example

#!/usr/bin/python3    

import smtplib    

sender_mail = '[email protected]'    

receivers_mail = ['[email protected]']    

message = """From: From Person %s  

To: To Person %s  

  

MIME-Version:1.0  

Content-type:text/html  

  

  

Subject: Sending SMTP e-mail   

  

<h3>Python SMTP</h3>  

<strong>This is a test e-mail message.</strong>  

"""%(sender_mail,receivers_mail)    

try:    

   smtpObj = smtplib.SMTP('localhost')    

   smtpObj.sendmail(sender_mail, receivers_mail, message)    

   print("Successfully sent email")    

except Exception:    

   print("Error: unable to send email")