A thread object goes through different stages during its life cycle. When a new thread object is created, it must be started, which calls the run() method of thread class. This method contains the logic of the process to be performed by the new thread. The thread completes its task as the run() method is over, and the newly created thread merges with the main thread.
While a thread is running, it may be paused either for a predefined duration or it may be asked to pause till a certain event occurs. The thread resumes after the specified interval or the process is over.
States of a Thread Life Cycle in Python
Following are the stages of the Python Thread life cycle −
Creating a Thread− To create a new thread in Python, you typically use the Thread class from the threading module.
Starting a Thread− Once a thread object is created, it must be started by calling its start() method. This initiates the thread’s activity and invokes its run() method in a separate thread.
Paused/Blocked State− Threads can be paused or blocked for various reasons, such as waiting for I/O operations to complete or another thread to perform a task. This is typically managed by calling its join() method. This blocks the calling thread until the thread being joined terminates.
Synchronizing Threads− Synchronization ensures orderly execution and shared resource management among threads. This can be done by using synchronization primitives like locks, semaphores, or condition variables.
Termination − A thread terminates when its run() method completes execution, either by finishing its task or encountering an exception.
Example: Python Thread Life Cycle Demonstration
This example demonstrates the thread life cycle in Python by showing thread creation, starting, execution, and synchronization with the main thread.
import threading
deffunc(x):print('Current Thread Details:', threading.current_thread())for n inrange(x):print('{} Running'.format(threading.current_thread().name), n)print('Internal Thread Finished...')# Create thread objects
t1 = threading.Thread(target=func, args=(2,))
t2 = threading.Thread(target=func, args=(3,))# Start the threadsprint('Thread State: CREATED')
t1.start()
t2.start()# Wait for threads to complete
t1.join()
t2.join()print('Threads State: FINISHED')# Simulate main thread workfor i inrange(3):print('Main Thread Running', i)print("Main Thread Finished...")
Output
When the above code is executed, it produces the following output −
Thread State: CREATED
Current Thread Details: <Thread(Thread-1 (func), started 140051032258112)>
Thread-1 (func) Running 0
Thread-1 (func) Running 1
Internal Thread Finished...
Current Thread Details: <Thread(Thread-2 (func), started 140051023865408)>
Thread-2 (func) Running 0
Thread-2 (func) Running 1
Thread-2 (func) Running 2
Internal Thread Finished...
Threads State: FINISHED
Main Thread Running 0
Main Thread Running 1
Main Thread Running 2
Main Thread Finished...
Example: Using a Synchronization Primitive
Here is another example demonstrates the thread life cycle in Python, including creation, starting, running, and termination states, along with synchronization using a semaphore.
import threading
import time
# Create a semaphore
semaphore = threading.Semaphore(2)defworker():with semaphore:print('{} has started working'.format(threading.current_thread().name))
time.sleep(2)print('{} has finished working'.format(threading.current_thread().name))# Create a list to keep track of thread objects
threads =[]# Create and start 5 threadsfor i inrange(5):
t = threading.Thread(target=worker, name='Thread-{}'.format(i+1))
threads.append(t)print('{} has been created'.format(t.name))
t.start()# Wait for all threads to completefor t in threads:
t.join()print('{} has terminated'.format(t.name))print('Threads State: All are FINISHED')print("Main Thread Finished...")
Output
When the above code is executed, it produces the following output −
Thread-1 has been created
Thread-1 has started working
Thread-2 has been created
Thread-2 has started working
Thread-3 has been created
Thread-4 has been created
Thread-5 has been created
Thread-1 has finished working
Thread-2 has finished working
Thread-3 has started working
Thread-1 has terminated
Thread-2 has terminated
Thread-4 has started working
Thread-3 has finished working
Thread-5 has started working
Thread-3 has terminated
Thread-4 has finished working
Thread-4 has terminated
Thread-5 has finished working
Thread-5 has terminated
Threads State: All are FINISHED
Main Thread Finished...
In Python, multithreading allows you to run multiple threads concurrently within a single process, which is also known as thread-based parallelism. This means a program can perform multiple tasks at the same time, enhancing its efficiency and responsiveness.
Multithreading in Python is especially useful for multiple I/O-bound operations, rather than for tasks that require heavy computation.
Generally, a computer program sequentially executes the instructions, from start to the end. Whereas, Multithreading divides the main task into more than one sub-task and executes them in an overlapping manner.
Comparison with Processes
An operating system is capable of handling multiple processes concurrently. It allocates a separate memory space to each process so that one process cannot access or write anything in other’s space.
On the other hand, a thread can be considered a lightweight sub-process in a single program that shares the memory space allocated to it, facilitating easier communication and data sharing. As they are lightweight and do not require much memory overhead; they are cheaper than processes.
A process always starts with a single thread (main thread). As and when required, a new thread can be started and sub task is delegated to it. Now the two threads are working in an overlapping manner. When the task assigned to the secondary thread is over, it merges with the main thread.
A thread has a beginning, an execution sequence, and a conclusion. It has an instruction pointer that keeps track of where it is currently running within its context.
It can be pre-empted (interrupted)
It can temporarily be put on hold (also known as sleeping) while other threads are running – this is called yielding.
Thread Handling Modules in Python
Python’s standard library provides two main modules for managing threads: _thread and threading.
The _thread Module
The _thread module, also known as the low-level thread module, has been a part of Python’s standard library since version 2. It offers a basic API for thread management, supporting concurrent execution of threads within a shared global data space. The module includes simple locks (mutexes) for synchronization purposes.
The threading Module
The threading module, introduced in Python 2.4, builds upon _thread to provide a higher-level and more comprehensive threading API. It offers powerful tools for managing threads, making it easier to work with threads in Python applications.
Key Features of the threading Module
The threading module exposes all the methods of the thread module and provides some additional methods −
threading.activeCount() Returns the number of thread objects that are active.
threading.currentThread() Returns the number of thread objects in the caller’s thread control.
threading.enumerate() Returns a list of all thread objects that are currently active.
In addition to the methods, the threading module has the Thread class that implements threading. The methods provided by the Thread class are as follows −
run() The run() method is the entry point for a thread.
start() The start() method starts a thread by calling the run method.
join([time]) The join() waits for threads to terminate.
isAlive() The isAlive() method checks whether a thread is still executing.
getName() The getName() method returns the name of a thread.
setName() The setName() method sets the name of a thread.
Starting a New Thread
To create and start a new thread in Python, you can use either the low-level _thread module or the higher-level threading module. The threading module is generally recommended due to its additional features and ease of use. Below, you can see both approaches.
Starting a New Thread Using the _thread Module
The start_new_thread() method of the _thread module provides a basic way to create and start new threads. This method provides a fast and efficient way to create new threads in both Linux and Windows. Following is the syntax of the method −
thread.start_new_thread(function, args[, kwargs])
This method call returns immediately, and the new thread starts executing the specified function with the given arguments. When the function returns, the thread terminates.
Example
This example demonstrates how to use the _thread module to create and run threads. Each thread runs the print_name function with different arguments. The time.sleep(0.5) call ensures that the main program waits for the threads to complete their execution before exiting.
import _thread
import time
defprint_name(name,*arg):print(name,*arg)
name="Tutorialspoint..."
_thread.start_new_thread(print_name,(name,1))
_thread.start_new_thread(print_name,(name,1,2))
time.sleep(0.5)
When the above code is executed, it produces the following result −
Tutorialspoint... 1
Tutorialspoint... 1 2
Although it is very effective for low-level threading, but the _thread module is limited compared to the threading module, which offers more features and higher-level thread management.
Starting a New Thread Using the Threading Module
The threading module provides the Thread class, which is used to create and manage threads.
Here are a few steps to start a new thread using the threading module −
Create a function that you want the thread to execute.
Then create a Thread object using the Thread class by passing the target function and its arguments.
Call the start method on the Thread object to begin execution.
Optionally, call the join method to wait for the thread to complete before proceeding.
Example
The following example demonstrates how to create and start threads using the threading module. It runs a function print_name that prints a name along with some arguments. This example creates two threads, starts them using the start() method, and waits for them to complete using the join method.
import threading
import time
defprint_name(name,*args):print(name,*args)
name ="Tutorialspoint..."# Create and start threads
thread1 = threading.Thread(target=print_name, args=(name,1))
thread2 = threading.Thread(target=print_name, args=(name,1,2))
thread1.start()
thread2.start()# Wait for threads to complete
thread1.join()
thread2.join()print("Threads are finished...exiting")
When the above code is executed, it produces the following result −
Tutorialspoint... 1
Tutorialspoint... 1 2
Threads are finished...exiting
Synchronizing Threads
The threading module provided with Python includes a simple-to-implement locking mechanism that allows you to synchronize threads. A new lock is created by calling the Lock() method, which returns the new lock.
The acquire(blocking) method of the new lock object is used to force threads to run synchronously. The optional blocking parameter enables you to control whether the thread waits to acquire the lock.
If blocking is set to 0, the thread returns immediately with a 0 value if the lock cannot be acquired and with a 1 if the lock was acquired. If blocking is set to 1, the thread blocks and wait for the lock to be released.
The release() method of the new lock object is used to release the lock when it is no longer required.
Example
import threading
import time
classmyThread(threading.Thread):def__init__(self, threadID, name, counter):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.counter = counter
defrun(self):print("Starting "+ self.name)# Get lock to synchronize threads
threadLock.acquire()
print_time(self.name, self.counter,3)# Free lock to release next thread
threadLock.release()defprint_time(threadName, delay, counter):while counter:
time.sleep(delay)print("%s: %s"%(threadName, time.ctime(time.time())))
counter -=1
threadLock = threading.Lock()
threads =[]# Create new threads
thread1 = myThread(1,"Thread-1",1)
thread2 = myThread(2,"Thread-2",2)# Start new Threads
thread1.start()
thread2.start()# Add threads to thread list
threads.append(thread1)
threads.append(thread2)# Wait for all threads to completefor t in threads:
t.join()print("Exiting Main Thread")</pre>
When the above code is executed, it produces the following result −
Starting Thread-1
Starting Thread-2
Thread-1: Thu Mar 21 09:11:28 2013
Thread-1: Thu Mar 21 09:11:29 2013
Thread-1: Thu Mar 21 09:11:30 2013
Thread-2: Thu Mar 21 09:11:32 2013
Thread-2: Thu Mar 21 09:11:34 2013
Thread-2: Thu Mar 21 09:11:36 2013
Exiting Main Thread
Multithreaded Priority Queue
The Queue module allows you to create a new queue object that can hold a specific number of items. There are following methods to control the Queue −
get() − The get() removes and returns an item from the queue.
put() − The put adds item to a queue.
qsize() − The qsize() returns the number of items that are currently in the queue.
empty() − The empty( ) returns True if queue is empty; otherwise, False.
full() − the full() returns True if queue is full; otherwise, False.
threadList =["Thread-1","Thread-2","Thread-3"]
nameList =["One","Two","Three","Four","Five"]
queueLock = threading.Lock()
workQueue = queue.Queue(10)
threads =[]
threadID =1# Create new threadsfor tName in threadList:
thread = myThread(threadID, tName, workQueue)
thread.start()
threads.append(thread)
threadID +=1# Fill the queue
queueLock.acquire()for word in nameList:
workQueue.put(word)
queueLock.release()# Wait for queue to emptywhilenot workQueue.empty():pass# Notify threads it's time to exit
exitFlag =1# Wait for all threads to completefor t in threads:
t.join()print("Exiting Main Thread")
When the above code is executed, it produces the following result −
Starting Thread-1
Starting Thread-2
Starting Thread-3
Thread-1 processing One
Thread-2 processing Two
Thread-3 processing Three
Thread-1 processing Four
Thread-2 processing Five
Exiting Thread-3
Exiting Thread-1
Exiting Thread-2
Exiting Main Thread
Built-in exceptions are pre-defined error classes in Python that handle errors and exceptional conditions in programs. They are derived from the base class “BaseException” and are part of the standard library.
Standard Built-in Exceptions in Python
Here is a list of Standard Exceptions available in Python −
Sr.No.
Exception Name & Description
1
ExceptionBase class for all exceptions
2
StopIterationRaised when the next() method of an iterator does not point to any object.
3
SystemExitRaised by the sys.exit() function.
4
StandardErrorBase class for all built-in exceptions except StopIteration and SystemExit.
5
ArithmeticErrorBase class for all errors that occur for numeric calculation.
6
OverflowErrorRaised when a calculation exceeds maximum limit for a numeric type.
7
FloatingPointErrorRaised when a floating point calculation fails.
8
ZeroDivisonErrorRaised when division or modulo by zero takes place for all numeric types.
9
AssertionErrorRaised in case of failure of the Assert statement.
10
AttributeErrorRaised in case of failure of attribute reference or assignment.
11
EOFErrorRaised when there is no input from either the raw_input() or input() function and the end of file is reached.
12
ImportErrorRaised when an import statement fails.
13
KeyboardInterruptRaised when the user interrupts program execution, usually by pressing Ctrl+C.
14
LookupErrorBase class for all lookup errors.
15
IndexErrorRaised when an index is not found in a sequence.
16
KeyErrorRaised when the specified key is not found in the dictionary.
17
NameErrorRaised when an identifier is not found in the local or global namespace.
18
UnboundLocalErrorRaised when trying to access a local variable in a function or method but no value has been assigned to it.
19
EnvironmentErrorBase class for all exceptions that occur outside the Python environment.
20
IOErrorRaised when an input/ output operation fails, such as the print statement or the open() function when trying to open a file that does not exist.
21
OSErrorRaised for operating system-related errors.
22
SyntaxErrorRaised when there is an error in Python syntax.
23
IndentationErrorRaised when indentation is not specified properly.
24
SystemErrorRaised when the interpreter finds an internal problem, but when this error is encountered the Python interpreter does not exit.
25
SystemExitRaised when Python interpreter is quit by using the sys.exit() function. If not handled in the code, causes the interpreter to exit.
26
TypeErrorRaised when an operation or function is attempted that is invalid for the specified data type.
27
ValueErrorRaised when the built-in function for a data type has the valid type of arguments, but the arguments have invalid values specified.
28
RuntimeErrorRaised when a generated error does not fall into any category.
29
NotImplementedErrorRaised when an abstract method that needs to be implemented in an inherited class is not actually implemented.
Here are some examples of standard exceptions −
IndexError
It is shown when trying to access item at invalid index.
numbers=[10,20,30,40]for n inrange(5):print(numbers[n])
It will produce the following output −
10
20
30
40
Traceback (most recent call last):
print (numbers[n])
IndexError: list index out of range
ModuleNotFoundError
This is displayed when module could not be found.
import notamodule
Traceback (most recent call last):import notamodule
ModuleNotFoundError: No module named 'notamodule'
This is encountered when object could not be found.
print(age)
Traceback (most recent call last):
age
NameError: name 'age'isnot defined
ZeroDivisionError
It is shown when second operator in division is zero.
x=100/0
Traceback (most recent call last):
x=100/0
ZeroDivisionError: division by zero
KeyboardInterrupt
When user hits the interrupt key normally Control-C during execution of program.
name=input('enter your name')
enter your name^c
Traceback (most recent call last):
name=input('enter your name')
KeyboardInterrupt
Hierarchy of Built-in Exceptions
The exceptions in Python are organized in a hierarchical structure, with “BaseException” at the top. Here is a simplified hierarchy −
BaseException
SystemExit
KeyboardInterrupt
Exception
ArithmeticError
FloatingPointError
OverflowError
ZeroDivisionError
AttributeError
EOFError
ImportError
LookupError
IndexError
KeyError
MemoryError
NameError
UnboundLocalError
OSError
FileNotFoundError
TypeError
ValueError
—(Many others)—
How to Use Built-in Exceptions
As we already know that built-in exceptions in Python are pre-defined classes that handle specific error conditions. Now, here is a detailed guide on how to use them effectively in your Python programs −
Handling Exceptions with try-except Blocks
The primary way to handle exceptions in Python is using “try-except” blocks. This allows you to catch and respond to exceptions that may occur during the execution of your code.
Example
In the following example, the code that may raise an exception is placed inside the “try” block. The “except” block catches the specified exception “ZeroDivisionError” and handles it
try:
result =1/0except ZeroDivisionError as e:print(f"Caught an exception: {e}")
Following is the output obtained −
Caught an exception: division by zero
Handling Multiple Exceptions
You can handle multiple exceptions by specifying them in a tuple within the “except” block as shown in the example below −
try:
result =int('abc')except(ValueError, TypeError)as e:print(f"Caught a ValueError or TypeError: {e}")
Output of the above code is as shown below −
Caught a ValueError or TypeError: invalid literal for int() with base 10: 'abc'
Using “else” and “finally” Blocks
The “else” block is executed if the code block in the “try” clause does not raise an exception −
try:
number =int(input("Enter a number: "))except ValueError as e:print(f"Invalid input: {e}")else:print(f"You entered: {number}")
Output of the above code varies as per the input given −
Enter a number: bn
Invalid input: invalid literal for int() with base 10: 'bn'
The “finally” block is always executed, regardless of whether an exception occurred or not. It’s typically used for clean-up actions, such as closing files or releasing resources −
try:file=open('example.txt','r')
content =file.read()except FileNotFoundError as e:print(f"File not found: {e}")finally:file.close()print("File closed.")
Following is the output of the above code −
File closed.
Explicitly Raising Built-in Exceptions
In Python, you can raise built-in exceptions to indicate errors or exceptional conditions in your code. This allows you to handle specific error scenarios and provide informative error messages to users or developers debugging your application.
Syntax
Following is the basic syntax for raising built-in exception −
raise ExceptionClassName("Error message")
Example
In this example, the “divide” function attempts to divide two numbers “a” and “b”. If “b” is zero, it raises a “ZeroDivisionError” with a custom message −
defdivide(a, b):if b ==0:raise ZeroDivisionError("Cannot divide by zero")return a / b
try:
result = divide(10,0)except ZeroDivisionError as e:print(f"Error: {e}")
Assertions in Python are statements that assert or assume a condition to be true. If the condition turns out to be false, Python raises an AssertionError exception. They are used to detect programming errors that should never occur if the code is correct.
The easiest way to think of an assertion is to liken it to a raise-if statement (or to be more accurate, a raise-if-not statement). An expression is tested, and if the result comes up false, an exception is raised.
Assertions are carried out by the assert statement, the newest keyword to Python, introduced in version 1.5.
Programmers often place assertions at the start of a function to check for valid input, and after a function call to check for valid output.
The assert Statement
In Python, assertions use the assert keyword followed by an expression. If the expression evaluates to False, an AssertionError is raised. Following is the syntax of assertion −
assert condition, message
Where,
condition − A boolean expression that should be true.
message (optional) − An optional message to be displayed if the assertion fails.
Using Assertions
Assertions are generally used during development and testing phases to check conditions that should always hold true.
Example
In the following example, we are using assertions to ensure that the variable “num” falls within the valid range of “0” to “100”. If the assertion fails, Python raises an “AssertionError”, preventing further execution of the subsequent print statement −
print('Enter marks out of 100:')
num =75assert num >=0and num <=100print('Marks obtained:', num)
num =125assert num >=0and num <=100print('Marks obtained:', num)# This line won't be reached if assertion fails
Following is the output of the above code −
Enter marks out of 100:
Marks obtained: 75
Traceback (most recent call last):
File "/home/cg/root/66723bd115007/main.py", line 7, in <module>
assert num >= 0 and num <= 100
AssertionError
Custom Error Messages
To display a custom error message when an assertion fails, include a string after the expression in the assert statement −
assert num >=0and num <=100,"Only numbers in the range 0-100 are accepted"
Handling AssertionError
Assertions can be caught and handled like any other exception using a try-except block. If they are not handled, they will terminate the program and produce a traceback −
try:
num =int(input('Enter a number: '))assert num >=0,"Only non-negative numbers are accepted"print(num)except AssertionError as msg:print(msg)
It will produce the following output −
Enter a number: -87
Only non-negative numbers are accepted
Assertions vs. Exceptions
Assertions are used to check internal state and invariants that should always be true. Whereas, exceptions helps in handling runtime errors and exceptional conditions that may occur during normal execution.
Assertions are disabled by default in Python’s optimized mode (-O or python -O script.py). Therefore, they should not be used to enforce constraints that are required for the correct functioning of the program in production environments.
Logging is the process of recording messages during the execution of a program to provide runtime information that can be useful for monitoring, debugging, and auditing.
In Python, logging is achieved through the built-in logging module, which provides a flexible framework for generating log messages.
Benefits of Logging
Following are the benefits of using logging in Python −
Debugging − Helps identify and diagnose issues by capturing relevant information during program execution.
Monitoring − Provides insights into the application’s behavior and performance.
Auditing − Keeps a record of important events and actions for security purposes.
Troubleshooting − Facilitates tracking of program flow and variable values to understand unexpected behavior.
Components of Python Logging
Python logging consists of several key components that work together to manage and output log messages effectively −
Logger − It is the main entry point that you use to emit log messages. Each logger instance is named and can be configured independently.
Handler − It determines where log messages are sent. Handlers send log messages to different destinations such as the console, files, sockets, etc.
Formatter − It specifies the layout of log messages. Formatters define the structure of log records by specifying which information to include (e.g., timestamp, log level, message).
Logger Level − It defines the severity level of log messages. Messages below this level are ignored. Common levels include DEBUG, INFO, WARNING, ERROR, and CRITICAL.
Filter − It is the optional components that provide finer control over which log records are processed and emitted by a handler.
Logging Levels
Logging levels in Python define the severity of log messages, allowing developers to categorize and filter messages based on their importance. Each logging level has a specific purpose and helps in understanding the significance of the logged information −
DEBUG − Detailed information, typically useful only for debugging purposes. These messages are used to trace the flow of the program and are usually not seen in production environments.
INFO − Confirmation that things are working as expected. These messages provide general information about the progress of the application.
WARNING − Indicates potential issues that do not prevent the program from running but might require attention. These messages can be used to alert developers about unexpected situations.
ERROR − Indicates a more serious problem that prevents a specific function or operation from completing successfully. These messages highlight errors that need immediate attention but do not necessarily terminate the application.
CRITICAL − The most severe level, indicating a critical error that may lead to the termination of the program. These messages are reserved for critical failures that require immediate intervention.
Usage
Following are the usage scenarios for each logging level in Python applications −
Choosing the Right Level − Selecting the appropriate logging level ensures that log messages provide relevant information without cluttering the logs.
Setting Levels − Loggers, handlers, and specific log messages can be configured with different levels to control which messages are recorded and where they are outputted.
Hierarchy − Logging levels are hierarchical, meaning that setting a level on a logger also affects the handlers and log messages associated with it.
Basic Logging Example
Following is a basic logging example in Python to demonstrate its usage and functionality −
import logging
# Configure logging
logging.basicConfig(level=logging.DEBUG,format='%(asctime)s - %(levelname)s - %(message)s')# Example usagedefcalculate_sum(a, b):
logging.debug(f"Calculating sum of {a} and {b}")
result = a + b
logging.info(f"Sum calculated successfully: {result}")return result
# Main programif __name__ =="__main__":
logging.info("Starting the program")
result = calculate_sum(10,20)
logging.info("Program completed")
Output
Following is the output of the above code −
2024-06-19 09:00:06,774 - INFO - Starting the program
2024-06-19 09:00:06,774 - DEBUG - Calculating sum of 10 and 20
2024-06-19 09:00:06,774 - INFO - Sum calculated successfully: 30
2024-06-19 09:00:06,775 - INFO - Program completed
Configuring Logging
Configuring logging in Python refers to setting up various components such as loggers, handlers, and formatters to control how and where log messages are stored and displayed. This configuration allows developers to customize logging behavior according to their application’s requirements and deployment environment.
Example
In the following example, the getLogger() function retrieves or creates a named logger. Loggers are organized hierarchically based on their names. Then, handlers like “StreamHandler” (console handler) are created to define where log messages go. They can be configured with specific log levels and formatters.
The formatters specify the layout of log records, determining how log messages appear when printed or stored −
import logging
# Create logger
logger = logging.getLogger('my_app')
logger.setLevel(logging.DEBUG)# Set global log level# Create console handler and set level to debug
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.DEBUG)# Create formatter
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
console_handler.setFormatter(formatter)# Add console handler to logger
logger.addHandler(console_handler)# Example usage
logger.debug('This is a debug message')
logger.info('This is an info message')
logger.warning('This is a warning message')
logger.error('This is an error message')
logger.critical('This is a critical message')
The result produced is as shown below −
2024-06-19 09:05:20,852 - my_app - DEBUG - This is a debug message
2024-06-19 09:05:20,852 - my_app - INFO - This is an info message
2024-06-19 09:05:20,852 - my_app - WARNING - This is a warning message
2024-06-19 09:05:20,852 - my_app - ERROR - This is an error message
2024-06-19 09:05:20,852 - my_app - CRITICAL - This is a critical message
Logging Handlers
Logging handlers in Python determine where and how log messages are processed and outputted. They play an important role in directing log messages to specific destinations such as the console, files, email, databases, or even remote servers.
Each handler can be configured independently to control the format, log level, and other properties of the messages it processes.
Types of Logging Handlers
Following are the various types of logging handlers in Python −
StreamHandler − Sends log messages to streams such as sys.stdout or sys.stderr. Useful for displaying log messages in the console or command line interface.
FileHandler − Writes log messages to a specified file on the file system. Useful for persistent logging and archiving of log data.
RotatingFileHandler − Similar to FileHandler but automatically rotates log files based on size or time intervals. Helps manage log file sizes and prevent them from growing too large.
SMTPHandler − Sends log messages as emails to designated recipients via SMTP. Useful for alerting administrators or developers about critical issues.
SysLogHandler − Sends log messages to the system log on Unix-like systems (e.g., syslog). Allows integration with system-wide logging facilities.
MemoryHandler − Buffers log messages in memory and sends them to a target handler after reaching a certain buffer size or timeout. Useful for batching and managing bursts of log messages.
HTTPHandler − Sends log messages to a web server via HTTP or HTTPS. Enables logging messages to a remote server or logging service.
User-defined exceptions in Python are custom error classes that you create to handle specific error conditions in your code. They are derived from the built-in Exception class or any of its sub classes.
User-defined exceptions provide more precise control over error handling in your application −
Clarity − They provide specific error messages that make it clear what went wrong.
Granularity − They allow you to handle different error conditions separately.
Maintainability − They centralize error handling logic, making your code easier to maintain.
How to Create a User-Defined Exception
To create a user-defined exception, follow these steps −
Step 1 − Define the Exception Class
Create a new class that inherits from the built-in “Exception” class or any other appropriate base class. This new class will serve as your custom exception.
classMyCustomError(Exception):pass
Explanation
Inheritance − By inheriting from “Exception”, your custom exception will have the same behaviour and attributes as the built-in exceptions.
Class Definition − The class is defined using the standard Python class syntax. For simple custom exceptions, you can define an empty class body using the “pass” statement.
Step 2 − Initialize the Exception
Implement the “__init__” method to initialize any attributes or provide custom error messages. This allows you to pass specific information about the error when raising the exception.
classInvalidAgeError(Exception):def__init__(self, age, message="Age must be between 18 and 100"):
self.age = age
self.message = message
super().__init__(self.message)</pre>
Explanation
Attributes − Define attributes such as "age" and "message" to store information about the error.
Initialization − The "__init__" method initializes these attributes. The "super().__init__(self.message)" call ensures that the base "Exception" class is properly initialized with the error message.
Default Message − A default message is provided, but you can override it when raising the exception.
Step 3 − Optionally Override "__str__" or "__repr__"
Override the "__str__" or "__repr__" method to provide a custom string representation of the exception. This is useful for printing or logging the exception.
classInvalidAgeError(Exception):def__init__(self, age, message="Age must be between 18 and 100"):
self.age = age
self.message = message
super().__init__(self.message)def__str__(self):returnf"{self.message}. Provided age: {self.age}"</pre>
Explanation
__str__ Method − The "__str__" method returns a string representation of the exception. This is what will be displayed when the exception is printed.
Custom Message − Customize the message to include relevant information, such as the provided age in this example.
Raising User-Defined Exceptions
Once you have defined a custom exception, you can raise it in your code to signify specific error conditions. Raising user-defined exceptions involves using the raisestatement, which can be done with or without custom messages and attributes.
Syntax
Following is the basic syntax for raising an exception −
raise ExceptionType(args)
Example
In this example, the "set_age" function raises an "InvalidAgeError" if the age is outside the valid range −
defset_age(age):if age <18or age >100:raise InvalidAgeError(age)print(f"Age is set to {age}")
Handling User-Defined Exceptions
Handling user-defined exceptions in Python refers to using "try-except" blocks to catch and respond to the specific conditions that your custom exceptions represent. This allows your program to handle errors gracefully and continue running or to take specific actions based on the type of exception raised.
Syntax
Following is the basic syntax for handling exceptions −
try:# Code that may raise an exceptionexcept ExceptionType as e:# Code to handle the exception
Example
In the below example, the "try" block calls "set_age" with an invalid age. The "except" block catches the "InvalidAgeError" and prints the custom error message −
try:
set_age(150)except InvalidAgeError as e:print(f"Invalid age: {e.age}. {e.message}")
Complete Example
Combining all the steps, here is a complete example of creating and using a user-defined exception −
classInvalidAgeError(Exception):def__init__(self, age, message="Age must be between 18 and 100"):
self.age = age
self.message = message
super().__init__(self.message)def__str__(self):returnf"{self.message}. Provided age: {self.age}"defset_age(age):if age <18or age >100:raise InvalidAgeError(age)print(f"Age is set to {age}")try:
set_age(150)except InvalidAgeError as e:print(f"Invalid age: {e.age}. {e.message}")
In a Python program, if there is another try-except construct either inside either a try block or inside its except block, it is known as a nested-try block. This is needed when different blocks like outer and inner may cause different errors. To handle them, we need nested try blocks.
We start with an example having a single “try − except − finally” construct. If the statements inside try encounter exception, it is handled by except block. With or without exception occurred, the finally block is always executed.
Example 1
Here, the try block has “division by 0” situation, hence the except block comes into play. It is equipped to handle the generic exception with Exception class.
Let us now see how to nest the try constructs. We put another “try − except − finally” blocks inside the existing try block. The except keyword for inner try now handles generic Exception, while we ask the except block of outer try to handle ZeroDivisionError.
Since exception doesn’t occur in the inner try block, its corresponding generic Except isn’t called. The division by 0 situation is handled by outer except clause.
a=10
b=0try:print(a/b)try:print("This is inner try block")except Exception:print("General exception")finally:print("inside inner finally block")except ZeroDivisionError:print("Division by 0")finally:print("inside outer finally block")
It will produce the following output −
Division by 0
inside outer finally block
Example 3
Now we reverse the situation. Out of the nested try blocks, the outer one doesn’t have any exception raised, but the statement causing division by 0 is inside inner try, and hence the exception handled by inner except block. Obviously, the except part corresponding to outer try: will not be called upon.
a=10
b=0try:print("This is outer try block")try:print(a/b)except ZeroDivisionError:print("Division by 0")finally:print("inside inner finally block")except Exception:print("General Exception")finally:print("inside outer finally block")
It will produce the following output −
This is outer try block
Division by 0
inside inner finally block
inside outer finally block
In the end, let us discuss another situation which may occur in case of nested blocks. While there isn’t any exception in the outer try:, there isn’t a suitable except block to handle the one inside the inner try: block.
Example 4
In the following example, the inner try: faces “division by 0”, but its corresponding except: is looking for KeyError instead of ZeroDivisionError. Hence, the exception object is passed on to the except: block of the subsequent except statement matching with outer try: statement. There, the zeroDivisionError exception is trapped and handled.
a=10
b=0try:print("This is outer try block")try:print(a/b)except KeyError:print("Key Error")finally:print("inside inner finally block")except ZeroDivisionError:print("Division by 0")finally:print("inside outer finally block")
It will produce the following output −
This is outer try block
inside inner finally block
Division by 0
inside outer finally block
Exception chaining is a technique of handling exceptions by re-throwing a caught exception after wrapping it inside a new exception. The original exception is saved as a property (such as cause) of the new exception.
During the handling of one exception ‘A’, it is possible that another exception ‘B’ may occur. It is useful to know about both exceptions in order to debug the problem. Sometimes it is useful for an exception handler to deliberately re-raise an exception, either to provide extra information or to translate an exception to another type.
In Python 3.x, it is possible to implement exception chaining. If there is any unhandled exception inside an except section, it will have the exception being handled attached to it and included in the error message.
Example
In the following code snippet, trying to open a non-existent file raises FileNotFoundError. It is detected by the except block. While handling another exception is raised.
try:open("nofile.txt")except OSError:raise RuntimeError("unable to handle error")
It will produce the following output −
Traceback (most recent call last):
File "/home/cg/root/64afcad39c651/main.py", line 2, in <module>
open("nofile.txt")
FileNotFoundError: [Errno 2] No such file or directory: 'nofile.txt'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/cg/root/64afcad39c651/main.py", line 4, in <module>
raise RuntimeError("unable to handle error")
RuntimeError: unable to handle error
The raise . . from Statement
If you use an optional from clause in the raise statement, it indicates that an exception is a direct consequence of another. This can be useful when you are transforming exceptions. The token after from keyword should be the exception object.
try:open("nofile.txt")except OSError as exc:raise RuntimeError from exc
It will produce the following output −
Traceback (most recent call last):
File "/home/cg/root/64afcad39c651/main.py", line 2, in <module>
open("nofile.txt")
FileNotFoundError: [Errno 2] No such file or directory: 'nofile.txt'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/cg/root/64afcad39c651/main.py", line 4, in <module>
raise RuntimeError from exc
RuntimeError
The raise . . from None Statement
If we use None in from clause instead of exception object, the automatic exception chaining that was found in the earlier example is disabled.
try:open("nofile.txt")except OSError as exc:raise RuntimeError fromNone
It will produce the following output −
Traceback (most recent call last):
File "C:\Python311\hello.py", line 4, in <module>
raise RuntimeError from None
RuntimeError
The __context__ and __cause__ Expression
Raising an exception in the except block will automatically add the captured exception to the __context__ attribute of the new exception. Similarly, you can also add __cause__ to any exception using the expression raise … from syntax.
try:try:raise ValueError("ValueError")except ValueError as e1:raise TypeError("TypeError")from e1
except TypeError as e2:print("The exception was",repr(e2))print("Its __context__ was",repr(e2.__context__))print("Its __cause__ was",repr(e2.__cause__))
It will produce the following output −
The exception was TypeError('TypeError')
Its __context__ was ValueError('ValueError')
Its __cause__ was ValueError('ValueError')
In Python, you can raise exceptions explicitly using the raise statement. Raising exceptions allows you to indicate that an error has occurred and to control the flow of your program by handling these exceptions appropriately.
Raising an exception refers to explicitly trigger an error condition in your program. This can be useful for handling situations where the normal flow of your program cannot continue due to an error or an unexpected condition.
In Python, you can raise built-in exceptions like ValueError or TypeError to indicate common error conditions. Additionally, you can create and raise custom exceptions.
Raising Built-in Exceptions
You can raise any built-in exception by creating an instance of the exception class and using the raise statement. Following is the syntax −
raise Exception("This is a general exception")
Example
Here is an example where we raise a ValueError when a function receives an invalid argument −
defdivide(a, b):if b ==0:raise ValueError("Cannot divide by zero")return a / b
try:
result = divide(10,0)except ValueError as e:print(e)
Following is the output of the above code −
Cannot divide by zero
Raising Custom Exceptions
In addition to built-in exceptions, you can define and raise your own custom exceptions by creating a new exception class that inherits from the base Exception class or any of its subclasses −
classMyCustomError(Exception):passdefrisky_function():raise MyCustomError("Something went wrong in risky_function")try:
risky_function()except MyCustomError as e:print(e)
Output of the above code is as shown below −
Something went wrong in risky_function
Creating Custom Exceptions
Custom exceptions is useful for handling specific error conditions that are unique to your application, providing more precise error reporting and control.
To create a custom exception in Python, you define a new class that inherits from the built-in Exception class or any other appropriate built-in exception class. This custom exception class can have additional attributes and methods to provide more detailed context about the error condition.
Example
In this example −
We define a custom exception class “InvalidAgeError” that inherits from “Exception”.
The __init__() method initializes the exception with the invalid age and a default error message.
The set_age() function raises “InvalidAgeError” if the provided age is outside the valid range.
classInvalidAgeError(Exception):def__init__(self, age, message="Age must be between 18 and 100"):
self.age = age
self.message = message
super().__init__(self.message)defset_age(age):if age <18or age >100:raise InvalidAgeError(age)print(f"Age is set to {age}")try:
set_age(150)except InvalidAgeError as e:print(f"Invalid age: {e.age}. {e.message}")
The result obtained is as shown below −
Invalid age: 150. Age must be between 18 and 100
Re-Raising Exceptions
Sometimes, you may need to catch an exception, perform specific actions (such as logging, cleanup, or providing additional context), and then re-raise the same exception to be handled further up the call stack
This is useful when you want to ensure certain actions are taken when an exception occurs, but still allow the exception to propagate for higher-level handling.
To re-raise an exception in Python, you use the “raise” statement without specifying an exception, which will re-raise the last exception that was active in the current scope.
Example
In the following example −
The process_file() function attempts to open and read a file.
If the file is not found, it prints an error message and re-raises the “FileNotFoundError” exception.
The exception is then caught and handled at a higher level in the call stack.
In Python, the try-finally block is used to ensure that certain code executes, regardless of whether an exception is raised or not. Unlike the try-except block, which handles exceptions, the try-finally block focuses on cleanup operations that must occur, ensuring resources are properly released and critical tasks are completed.
Syntax
The syntax of the try-finally statement is as follows −
try:# Code that might raise exceptions
risky_code()finally:# Code that always runs, regardless of exceptions
cleanup_code()
In Python, when using exception handling with try blocks, you have the option to include either except clauses to catch specific exceptions or a finally clause to ensure certain cleanup operations are executed, but not both together.
Example
Let us consider an example where we want to open a file in write mode (“w”), writes some content to it, and ensures the file is closed regardless of success or failure using a finally block −
try:
fh =open("testfile","w")
fh.write("This is my test file for exception handling!!")finally:print("Error: can\'t find file or read data")
fh.close()
If you do not have permission to open the file in writing mode, then it will produce the following output −
Error: can't find file or read data
The same example can be written more cleanly as follows −
try:
fh =open("testfile","w")try:
fh.write("This is my test file for exception handling!!")finally:print("Going to close the file")
fh.close()except IOError:print("Error: can\'t find file or read data")</pre>
When an exception is thrown in the try block, the execution immediately passes to the finally block. After all the statements in the finally block are executed, the exception is raised again and is handled in the except statements if present in the next higher layer of the try-except statement.
Exception with Arguments
An exception can have an argument, which is a value that gives additional information about the problem. The contents of the argument vary by exception. You capture an exception's argument by supplying a variable in the except clause as follows −
try:
You do your operations here
......................except ExceptionType as Argument:
You can print value of Argument here...
If you write the code to handle a single exception, you can have a variable follow the name of the exception in the except statement. If you are trapping multiple exceptions, you can have a variable follow the tuple of the exception.
This variable receives the value of the exception mostly containing the cause of the exception. The variable can receive a single value or multiple values in the form of a tuple. This tuple usually contains the error string, the error number, and an error location.
Example
Following is an example for a single exception −
# Define a function here.deftemp_convert(var):try:returnint(var)except ValueError as Argument:print("The argument does not contain numbers\n",Argument)# Call above function here.
temp_convert("xyz")
It will produce the following output −
The argument does not contain numbers
invalid literal for int() with base 10: 'xyz'
