In Python, generics is a mechanism with which you to define functions, classes, or methods that can operate on multiple types while maintaining type safety. With the implementation of Generics enable it is possible to write reusable code that can be used with different data types. It ensures promoting code flexibility and type correctness.
Normally, in Python programming, you don’t need to declare a variable type. The type is determined dynamically by the value assigned to it. Python’s interpreter doesn’t perform type checks and hence it may raise runtime exceptions.
Python introduced generics with type hints in version 3.5, allowing you to specify the expected types of variables, function arguments, and return values. This feature helps in reducing runtime errors and improving code readability.
Generics extend the concept of type hints by introducing type variables, which represent generic types that can be replaced with specific types when using the generic function or class.
Defining a Generic Function
Let us have a look at the following example that defines a generic function −
from typing import List, TypeVar, Generic
T = TypeVar('T')defreverse(items: List[T])-> List[T]:return items[::-1]
Here, we define a generic function called ‘reverse’. The function takes a list (‘List[T]’) as an argument and returns a list of the same type. The type variable ‘T’ represents the generic type, which will be replaced with a specific type when the function is used.
Calling the Generic Function with Different Data Types
The function reverse() function is called with different data types −
A generic type is typically declared by adding a list of type parameters after the class name. The following example uses generics with a generic class −
from typing import List, TypeVar, Generic
T = TypeVar('T')classBox(Generic[T]):def__init__(self, item: T):
self.item = item
defget_item(self)-> T:return self.item
Let us create objects of the above generic classwithintandstrtype
box1 = Box(42)print(box1.get_item())
box2 = Box('Hello')print(box2.get_item())
In the world of Internet, different resources are identified by URLs (Uniform Resource Locators). Python’s standard library includes the urllib package, which has modules for working with URLs. It helps you parse URLs, fetch web content, and manage errors.
This tutorial introduces urllib basics to help you start using it. Improve your skills in web scraping, fetching data, and managing URLs with Python using urllib.
The urllib package contains the following modules for processing URLs −
urllib.parse module is used for parsing a URL into its parts.
urllib.request module contains functions for opening and reading URLs
urllib.error module carries definitions of the exceptions raised by urllib.request
urllib.robotparser module parses the robots.txt files
The urllib.parse Module
This module serves as a standard interface to obtain various parts from a URL string. The module contains following functions −
urlparse(urlstring)
Parse a URL into six components, returning a 6-item named tuple. Each tuple item is a string corresponding to following attributes −
This function Parse a query string given as a string argument. Data is returned as a dictionary. The dictionary keys are the unique query variable names and the values are lists of values for each name.
To further fetch the query parameters from the query string into a dictionary, use parse_qs() function of the query attribute of ParseResult object as follows −
This is similar to urlparse(), but does not split the params from the URL. This should generally be used instead of urlparse() if the more recent URL syntax allowing parameters to be applied to each segment of the path portion of the URL is wanted.
urlunparse(parts)
This function is the opposite of urlparse() function. It constructs a URL from a tuple as returned by urlparse(). The parts argument can be any six-item iterable. This returns an equivalent URL.
Example
from urllib.parse import urlunparse
lst =['https','example.com','/employees/name/','','salary>=25000','']
new_url = urlunparse(lst)print("URL:", new_url)
Combine the elements of a tuple as returned by urlsplit() into a complete URL as a string. The parts argument can be any five-item iterable.
The urllib.request Module
This module offers the functions and classes for handling the URL’s opening and reading operations by using the urlopen() function.
urlopen() function
This function opens the given URL, which can be either a string or a Request object. The optional timeout parameter specifies a timeout in seconds for blocking operations This actually only works for HTTP, HTTPS and FTP connections.
This function always returns an object which can work as a context manager and has the properties url, headers, and status. For HTTP and HTTPS URLs, this function returns a http.client.HTTPResponse object slightly modified.
Example
The following code uses urlopen() function to read the binary data from an image file, and writes it to local file. You can open the image file on your computer using any image viewer.
from urllib.request import urlopen
obj = urlopen("https://www.tutorialspoint.com/images/logo.png")
data = obj.read()
img =open("img.jpg","wb")
img.write(data)
img.close()
It will produce the following output −
The Request Object
The urllib.request module includes Request class. This class is an abstraction of a URL request. The constructor requires a mandatory string argument a valid URL.
data − An object specifying additional data to send to the server. This parameter can only be used with HTTP requests. Data may be bytes, file-like objects, and iterables of bytes-like objects.
headers − Should be a dictionary of headers and their associated values.
origin_req_host − Should be the request-host of the origin transaction
method − should be a string that indicates the HTTP request method. One of GET, POST, PUT, DELETE and other HTTP verbs. Default is GET.
Example
from urllib.request import Request
obj = Request("https://www.tutorialspoint.com/")
This Request object can now be used as an argument to urlopen() method.
Following exceptions are defined in urllib.error module −
URLError
URLError is raised because there is no network connection (no route to the specified server), or the specified server doesn’t exist. In this case, the exception raised will have a ‘reason’ attribute.
Example
from urllib.request import Request, urlopen
import urllib.error as err
obj = Request("http://www.nosuchserver.com")try:
urlopen(obj)except err.URLError as e:print(e)
It will produce the following output −
HTTP Error 403: Forbidden
HTTPError
Every time the server sends a HTTP response it is associated with a numeric “status code”. It code indicates why the server is unable to fulfill the request. The default handlers will handle some of these responses for you. For those it can’t handle, urlopen() function raises an HTTPError. Typical examples of HTTPErrors are ‘404’ (page not found), ‘403’ (request forbidden), and ‘401’ (authentication required).
Example
from urllib.request import Request, urlopen
import urllib.error as err
obj = Request("http://www.python.org/fish.html")try:
urlopen(obj)except err.HTTPError as e:print(e.code)
Socket programming is a technique in which we communicate between two nodes connected in a network where the server node listens to the incoming requests from the client nodes.
In Python, the socket module is used for socket programming. The socketmodule in the standard library included functionality required for communication between server and client at hardware level.
This module provides access to the BSD socket interface. It is available on all operating systems such as Linux, Windows, MacOS.
What are Sockets?
Sockets are the endpoints of a bidirectional communications channel. Sockets may communicate within a process, between processes on the same machine, or between processes on different continents.
A socket is identified by the combination of IP address and the port number. It should be properly configured at both ends to begin communication.
Sockets may be implemented over a number of different channel types: Unix domain sockets, TCP, UDP, and so on. The socket library provides specific classes for handling the common transports as well as a generic interface for handling the rest.
The term socket programming implies programmatically setting up sockets to be able to send and receive data.
There are two types of communication protocols −
connection-oriented protocol
connection-less protocol
TCP or Transmission Control Protocol is a connection-oriented protocol. The data is transmitted in packets by the server, and assembled in the same order of transmission by the receiver. Since the sockets at either end of the communication need to be set before starting, this method is more reliable.
UDP or User Datagram Protocol is connectionless. The method is not reliable because the sockets does not require establishing any connection and termination process for transferring the data.
Python socket Module
The socket module is used for creating and managing socket programming for the connected nodes in a network. The socket module provides a socket class. You need to create a socket using the socket.socket() constructor.
An object of the socket class represents the pair of host name and the port numbers.
Syntax
The following is the syntax of socket.socket() constructor –
family − AF_INET by default. Other values – AF_INET6 (eight groups of four hexadecimal digits), AF_UNIX, AF_CAN (Controller Area Network) or AF_RDS (Reliable Datagram Sockets).
socket_type − should be SOCK_STREAM (the default), SOCK_DGRAM, SOCK_RAW or perhaps one of the other SOCK_ constants.
protocol − number is usually zero and may be omitted.
Return Type
This method returns a socket object.
Once you have the socket object, then you can use the required methods to create your client or server program.
Server Socket Methods
The socket instantiated on server is called server socket. Following methods are available to the socket object on the server −
bind() method − This method binds the socket to specified IP address and port number.
listen() method − This method starts server and runs into a listen loop looking for connection request from client.
accept() method − When connection request is intercepted by server, this method accepts it and identifies the client socket with its address.
To create a socket on a server, use the following snippet −
By default, the server is bound to local machine’s IP address ‘localhost’ listening at arbitrary empty port number.
Client Socket Methods
Similar socket is set up on the client end. It mainly sends connection request to server socket listening at its IP address and port number
connect() method
This method takes a two-item tuple object as argument. The two items are IP address and port number of the server.
obj=socket.socket()
obj.connect((host,port))
Once the connection is accepted by the server, both the socket objects can send and/or receive data.
send() method
The server sends data to client by using the address it has intercepted.
client.send(bytes)
Client socket sends data to socket it has established connection with.
sendall() method
similar to send(). However, unlike send(),this method continues to send data from bytes until either all data has been sent or an error occurs. None is returned on success.
sendto() method
This method is to be used in case of UDP protocol only.
recv() method
This method is used to retrieve data sent to the client. In case of server, it uses the remote socket whose request has been accepted.
client.recv(bytes)
recvfrom() method
This method is used in case of UDP protocol.
Python – Socket Server
To write Internet servers, we use the socket function available in socket module to create a socket object. A socket object is then used to call other functions to setup a socket server.
Now call the bind(hostname, port) function to specify a port for your service on the given host.
Next, call the accept method of the returned object. This method waits until a client connects to the port you specified, and then returns a connection object that represents the connection to that client.
Example of Server Socket
import socket
host ="127.0.0.1"
port =5001
server = socket.socket()
server.bind((host,port))
server.listen()
conn, addr = server.accept()print("Connection from: "+str(addr))whileTrue:
data = conn.recv(1024).decode()ifnot data:break
data =str(data).upper()print(" from client: "+str(data))
data =input("type message: ")
conn.send(data.encode())
conn.close()
Python – Socket Client
Let us write a very simple client program, which opens a connection to a given port 5001 and a given localhost. It is very simple to create a socket client using the Python’s socket module function.
The socket.connect(hosname, port) opens a TCP connection to hostname on the port. Once you have a socket open, you can read from it like any IO object. When done, remember to close it, as you would close a file.
Example of Client Socket
The following code is a very simple client that connects to a given host and port, reads any available data from the socket, and then exits when ‘q’ is entered.
import socket
host ='127.0.0.1'
port =5001
obj = socket.socket()
obj.connect((host,port))
message =input("type message: ")while message !='q':
obj.send(message.encode())
data = obj.recv(1024).decode()print('Received from server: '+ data)
message =input("type message: ")
obj.close()
Run Server code first. It starts listening.
Then start client code. It sends request.
Request accepted. Client address identified
Type some text and press Enter.
Data received is printed. Send data to client.
Data from server is received.
Loop terminates when ‘q’ is input.
Server-client interaction is shown below −
We have implemented client-server communication with socket module on the local machine. To put server and client codes on two different machines on a network, we need to find the IP address of the server machine.
On Windows, you can find the IP address by running ipconfig command. The ifconfig command is the equivalent command on Ubuntu.
Change host string in both the server and client codes with IPv4 Address value and run them as before.
Python File Transfer with Socket Module
The following program demonstrates how socket communication can be used to transfer a file from server to the client
Server Code
The code for establishing connection is same as before. After the connection request is accepted, a file on server is opened in binary mode for reading, and bytes are successively read and sent to the client stream till end of file is reached.
import socket
host ="127.0.0.1"
port =5001
server = socket.socket()
server.bind((host, port))
server.listen()
conn, addr = server.accept()
data = conn.recv(1024).decode()
filename='test.txt'
f =open(filename,'rb')whileTrue:
l = f.read(1024)ifnot l:break
conn.send(l)print('Sent ',repr(l))
f.close()print('File transferred')
conn.close()
Client Code
On the client side, a new file is opened in wb mode. The stream of data received from server is written to the file. As the stream ends, the output file is closed. A new file will be created on the client machine.
import socket
s = socket.socket()
host ="127.0.0.1"
port =5001
s.connect((host, port))
s.send("Hello server!".encode())withopen('recv.txt','wb')as f:whileTrue:print('receiving data...')
The socketserver module in Python’s standard library is a framework for simplifying task of writing network servers. There are following classes in module, which represent synchronous servers −
These classes work with corresponding RequestHandler classes for implementing the service. BaseServer is the superclass of all Server objects in the module.
TCPServer class uses the internet TCP protocol, to provide continuous streams of data between the client and server. The constructor automatically attempts to invoke server_bind() and server_activate(). The other parameters are passed to the BaseServer base class.
You must also create a subclass of StreamRequestHandler class. IT provides self.rfile and self.wfile attributes to read or write to get the request data or return data to the client.
UDPServer and DatagramRequestHandler − These classes are meant to be used for UDP protocol.
DatagramRequestHandler and UnixDatagramServer − These classes use Unix domain sockets; they’re not available on non-Unix platforms.
Server Code
You must write a RequestHandler class. It is instantiated once per connection to the server, and must override the handle() method to implement communication to the client.
On the server's assigned port number, an object of TCPServer class calls the forever() method to put the server in the listening mode and accepts incoming requests from clients.
if __name__ =="__main__":
HOST, PORT ="localhost",9999with socketserver.TCPServer((HOST, PORT), MyTCPHandler)as server:
server.serve_forever()</pre>
Client Code
When working with socketserver, the client code is more or less similar with the socket client application.
import socket
import sys
HOST, PORT ="localhost",9999whileTrue:with socket.socket(socket.AF_INET, socket.SOCK_STREAM)as sock:# Connect to server and send data
sock.connect((HOST, PORT))
data =input("enter text .. .")
sock.sendall(bytes(data +"\n","utf-8"))# Receive data from the server and shut down
received =str(sock.recv(1024),"utf-8")print("Sent: {}".format(data))print("Received: {}".format(received))</pre>
Run the server code in one command prompt terminal. Open multiple terminals for client instances. You can simulate a concurrent communication between the server and more than one clients.
Server
Client-1
Client-2
D:\socketsrvr>python myserver.py127.0.0.1:54518 wrote:from client-1enter text ..hello127.0.0.1:54522 wrote:how are youenter text ..fine127.0.0.1:54523 wrote:from client-2enter text ..hi client-2127.0.0.1:54526 wrote:good byeenter text ..bye bye127.0.0.1:54530 wrote:thanksenter text ..bye client-2
The threading module in Python’s standard library is capable of handling multiple threads and their interaction within a single process. Communication between two processes running on the same machine is handled by Unix domain sockets, whereas for the processes running on different machines connected with TCP (Transmission control protocol), Internet domain sockets are used.
Python’s standard library consists of various built-in modules that support interprocess communication and networking. Python provides two levels of access to the network services. At a low level, you can access the basic socket support in the underlying operating system, which allows you to implement clients and servers for both connection-oriented and connectionless protocols.
Python also has libraries that provide higher-level access to specific application-level network protocols, such as FTP, HTTP, and so on.
