Blog

  • Type Hints

    Python type hints were introduced in PEP 484 to bring the benefits of static typing to a dynamically typed language. Although type hints do not enforce type checking at runtime, they provide a way to specify the expected types of variables, function parameters, and return values, which can be checked by static analysis tools such as mypy. This enhances code readability, facilitates debugging, and improves the overall maintainability of the code.

    Type hints in Python use annotations for function parameters, return values and variable assignments.

    Python’s type hints can be used to specify a wide variety of types such as basic data types, collections, complex types and custom user-defined types. The typing module provides many built-in types to represent these various types −

    Let’s see each one, one after another in detail.

    Basic Data Types

    In Python when using type hints to specify basic types we can simply use the name of the type as the annotation.

    Example

    Following is the example of using the basic data types such as integer, float, string etc −

    Open Compiler

    from typing import Optional
    
    # Integer typedefcalculate_square_area(side_length:int)->int:return side_length **2# Float typedefcalculate_circle_area(radius:float)->float:return3.14* radius * radius
    
    # String typedefgreet(name:str)->str:returnf"Hello, {name}"# Boolean typedefis_adult(age:int)->bool:return age >=18# None typedefno_return_example()->None:print("This function does not return anything")# Optional type (Union of int or None)defsafe_divide(x:int, y: Optional[int])-> Optional[float]:if y isNoneor y ==0:returnNoneelse:return x / y
    
    # Example usageprint(calculate_square_area(5))print(calculate_circle_area(3.0))print(greet("Alice"))print(is_adult(22))                   
    no_return_example()print(safe_divide(10,2))print(safe_divide(10,0))print(safe_divide(10,None))

    On executing the above code we will get the following output −

    25
    28.259999999999998
    Hello, Alice
    True
    This function does not return anything
    5.0
    None
    None
    

    Collections Types

    In Python when dealing with collections such as liststuplesdictionaries, etc. in type hints we typically use the typing module to specify the collection types.

    Example

    Below is the example of the Collections using in type hints −

    Open Compiler

    from typing import List, Tuple, Dict, Set, Iterable, Generator
    
    # List of integersdefprocess_numbers(numbers: List[int])-> List[int]:return[num *2for num in numbers]# Tuple of floatsdefcoordinates()-> Tuple[float,float]:return(3.0,4.0)# Dictionary with string keys and integer valuesdeffrequency_count(items: List[str])-> Dict[str,int]:
       freq ={}for item in items:
    
      freq[item]= freq.get(item,0)+1return freq
    # Set of unique characters in a stringdefunique_characters(word:str)-> Set[str]:returnset(word)# Iterable of integersdefprint_items(items: Iterable[int])->None:for item in items:print(item)# Generator yielding squares of integers up to ndefsquares(n:int)-> Generator[int,None,None]:for i inrange(n):yield i * i # Example usage numbers =[1,2,3,4,5]print(process_numbers(numbers))print(coordinates()) items =["apple","banana","apple","orange"]print(frequency_count(items)) word ="hello"print(unique_characters(word)) print_items(range(5)) gen = squares(5)print(list(gen))

    On executing the above code we will get the following output −

    [2, 4, 6, 8, 10]
    (3.0, 4.0)
    {'apple': 2, 'banana': 1, 'orange': 1}
    {'l', 'e', 'h', 'o'}
    0
    1
    2
    3
    4
    [0, 1, 4, 9, 16]
    

    Optional Types

    In Python, Optional types are used to indicate that a variable can either be of a specified type or None. This is particularly useful when a function may not always return a value or when a parameter can accept a value or be left unspecified.

    Example

    Here is the example of using the optional types in type hints −

    Open Compiler

    from typing import Optional
    
    defdivide(a:float, b:float)-> Optional[float]:if b ==0:returnNoneelse:return a / b
    
    result1: Optional[float]= divide(10.0,2.0)# result1 will be 5.0
    result2: Optional[float]= divide(10.0,0.0)# result2 will be Noneprint(result1)print(result2)

    On executing the above code we will get the following output −

    5.0
    None
    

    Union Types

    Python uses Union types to allow a variable to accept values of different types. This is useful when a function or data structure can work with various types of inputs or produce different types of outputs.

    Example

    Below is the example of this −

    Open Compiler

    from typing import Union
    
    defsquare_root_or_none(number: Union[int,float])-> Union[float,None]:if number >=0:return number **0.5else:returnNone
    
    result1: Union[float,None]= square_root_or_none(50)   
    result2: Union[float,None]= square_root_or_none(-50)print(result1)print(result2)

    On executing the above code we will get the following output −

    7.0710678118654755
    None
    

    Any Type

    In Python, Any type is a special type hint that indicates that a variable can be of any type. It essentially disables type checking for that particular variable or expression. This can be useful in situations where the type of a value is not known beforehand or when dealing with dynamic data.

    Example

    Following is the example of using Any type in Type hint −

    Open Compiler

    from typing import Any
    
    defprint_value(value: Any)->None:print(value)
    
    print_value(10)         
    print_value("hello")    
    print_value(True)       
    print_value([1,2,3])  
    print_value({'key':'value'})

    On executing the above code we will get the following output −

    10
    hello
    True
    [1, 2, 3]
    {'key': 'value'}
    

    Type Aliases

    Type aliases in Python are used to give alternative names to existing types. They can make code easier to read by giving clear names to complicated type annotations or combinations of types. This is especially helpful when working with nested structures or long-type hints.

    Example

    Below is the example of using the Type Aliases in the Type hints −

    Open Compiler

    from typing import List, Tuple
    
    # Define a type alias for a list of integers
    Vector = List[int]# Define a type alias for a tuple of coordinates
    Coordinates = Tuple[float,float]# Function using the type aliasesdefscale_vector(vector: Vector, factor:float)-> Vector:return[int(num * factor)for num in vector]defcalculate_distance(coord1: Coordinates, coord2: Coordinates)->float:
       x1, y1 = coord1
       x2, y2 = coord2
       return((x2 - x1)**2+(y2 - y1)**2)**0.5# Using the type aliases
    v: Vector =[1,2,3,4]
    scaled_v: Vector = scale_vector(v,2.5)print(scaled_v)  
    
    c1: Coordinates =(3.0,4.0)
    c2: Coordinates =(6.0,8.0)
    distance:float= calculate_distance(c1, c2)print(distance)

    On executing the above code we will get the following output −

    [2, 5, 7, 10]
    5.0
    

    Generic Types

    Generic types create functions, classes or data structures that can handle any type while maintaining type safety. The typing module’s TypeVar and Generic constructs make this possible. They are helpful for making reusable components that can work with various types without compromising type checking.

    Example

    Here is the example of it −

    Open Compiler

    from typing import TypeVar, List
    
    # Define a type variable T
    T = TypeVar('T')# Generic function that returns the first element of a listdeffirst_element(items: List[T])-> T:return items[0]# Example usage
    int_list =[1,2,3,4,5]
    str_list =["apple","banana","cherry"]
    
    first_int = first_element(int_list)# first_int will be of type int
    first_str = first_element(str_list)# first_str will be of type strprint(first_int)print(first_str)

    On executing the above code we will get the following output −

    1
    apple
    

    Callable Types

    Python’s Callable type is utilized to show that a type is a function or a callable object. It is found in the typing module and lets you define the types of the arguments and the return type of a function. This is handy for higher-order functions.

    Example

    Following is the example of using Callable type in type hint −

    Open Compiler

    from typing import Callable
    
    # Define a function that takes another function as an argumentdefapply_operation(x:int, y:int, operation: Callable[[int,int],int])->int:return operation(x, y)# Example functions to be passed as argumentsdefadd(a:int, b:int)->int:return a + b
    
    defmultiply(a:int, b:int)->int:return a * b
    
    # Using the apply_operation function with different operations
    result1 = apply_operation(5,3, add)# result1 will be 8
    result2 = apply_operation(5,3, multiply)# result2 will be 15print(result1)print(result2)

    On executing the above code we will get the following output −

    8
    15
    

    Literal Types

    The Literal type is used to specify that a value must be exactly one of a set of predefined values.

    Example

    Below is the example −

    from typing import Literal
    
    defmove(direction: Literal["left","right","up","down"])->None:print(f"Moving {direction}")
    
    move("left")# Valid
    move("up")# Valid                                     

    On executing the above code we will get the following output −

    Moving left
    Moving up
    

    NewType

    NewType is a function in the typing module that allows us to create distinct types derived from existing ones. This can be useful for adding type safety to our code by distinguishing between different uses of the same underlying type. For example we might want to differentiate between user IDs and product IDs even though both are represented as integers.

    Example

    Below is the example −

    from typing import NewType
    
    # Create new types
    UserId = NewType('UserId',int)
    ProductId = NewType('ProductId',int)# Define functions that use the new typesdefget_user_name(user_id: UserId)->str:returnf"User with ID {user_id}"defget_product_name(product_id: ProductId)->str:returnf"Product with ID {product_id}"# Example usage
    user_id = UserId(42)
    product_id = ProductId(101)print(get_user_name(user_id))# Output: User with ID 42print(get_product_name(product_id))# Output: Product with ID 101                                   

    On executing the above code we will get the following output −

    User with ID 42
    Product with ID 101
  •  Signal Handling

    Signal handling in Python allows you to define custom handlers for managing asynchronous events such as interrupts or termination requests from keyboard, alarms, and even system signals. You can control how your program responds to various signals by defining custom handlers. The signal module in Python provides mechanisms to set and manage signal handlers.

    signal handler is a function that gets executed when a specific signal is received. The signal.signal() function allows defining custom handlers for signals. The signal module offers a way to define custom handlers that will be executed when a specific signal is received. Some default handlers are already installed in Python, which are −

    • SIGPIPE is ignored.
    • SIGINT is translated into a KeyboardInterrupt exception.

    Commonly Used Signals

    Python signal handlers are executed in the main Python thread of the main interpreter, even if the signal is received in another thread. Signals can’t be used for inter-thread communication.

    Following are the list of some common signals and their default actions −

    • SIGINT − Interrupt from keyboard (Ctrl+C), which raises a KeyboardInterrupt.
    • SIGTERM − Termination signal.
    • SIGALRM− Timer signal from alarm().
    • SIGCHLD − Child process stopped or terminated.
    • SIGUSR1 and SIGUSR2 − User-defined signals.

    Setting a Signal Handler

    To set a signal handler, we can use the signal.signal() function. It allows you to define custom handlers for signals. A handler remains installed until explicitly reset, except for SIGCHLD.

    Example

    Here is an example of setting a signal handler using the signal.signal() function with the SIGINT handler.

    import signal
    import time
    
    defhandle_signal(signum, frame):print(f"Signal {signum} received")# Setting the handler for SIGINT
    signal.signal(signal.SIGINT, handle_signal)print("Press Ctrl+C to trigger SIGINT")whileTrue:
       time.sleep(1)

    Output

    On executing the above program, you will get the following results −

    Press Ctrl+C to trigger SIGINT
    Signal 2 received
    Signal 2 received
    Signal 2 received
    Signal 2 received
    

    Signal Handling on Windows

    On Windows, the signal.signal() function can only handle a limited set of signals. If you try to use a signal not supported on Windows, a ValueError will be raised. And, an AttributeError will be raised if a signal name is not defined as a SIG* module level constant.

    The supported signals on Windows are follows −

    • SIGABRT
    • SIGFPE
    • SIGILL
    • SIGINT
    • SIGSEGV
    • SIGTERM
    • SIGBREAK

    Handling Timers and Alarms

    Timers and alarms can be used to schedule signal delivery after a certain amount of time.

    Example

    Let’s observe following example of handling alarms.

    import signal
    import time
    
    defhandler(signum, stack):print('Alarm: ', time.ctime())
    
    signal.signal(signal.SIGALRM, handler)
    signal.alarm(2)
    time.sleep(5)for i inrange(5):
       signal.alarm(2)
       time.sleep(5)print("interrupted #%d"% i)

    Output

    On executing the above program, you will get the following results −

    Alarm:  Wed Jul 17 17:30:11 2024
    Alarm:  Wed Jul 17 17:30:16 2024
    interrupted #0
    Alarm:  Wed Jul 17 17:30:21 2024
    interrupted #1
    Alarm:  Wed Jul 17 17:30:26 2024
    interrupted #2
    Alarm:  Wed Jul 17 17:30:31 2024
    interrupted #3
    Alarm:  Wed Jul 17 17:30:36 2024
    interrupted #4
    

    Getting Signal Names from Numbers

    There is no straightforward way of getting signal names from numbers in Python. You can use the signal module to get all its attributes, filter out those that start with SIG, and store them in a dictionary.

    Example

    This example creates a dictionary where the keys are signal numbers and the values are the corresponding signal names. This is useful for dynamically resolving signal names from their numeric values.

    import signal
    
    sig_items =reversed(sorted(signal.__dict__.items()))
    final =dict((k, v)for v, k in sig_items if v.startswith('SIG')andnot v.startswith('SIG_'))print(final)

    Output

    On executing the above program, you will get the following results −

    {<Signals.SIGXFSZ: 25>: 'SIGXFSZ', <Signals.SIGXCPU: 24>: 'SIGXCPU', <Signals.SIGWINCH: 28>: 'SIGWINCH', <Signals.SIGVTALRM: 26>: 'SIGVTALRM', <Signals.SIGUSR2: 12>: 'SIGUSR2', <Signals.SIGUSR1: 10>: 'SIGUSR1', <Signals.SIGURG: 23>: 'SIGURG', <Signals.SIGTTOU: 22>: 'SIGTTOU', <Signals.SIGTTIN: 21>: 'SIGTTIN', <Signals.SIGTSTP: 20>: 'SIGTSTP', <Signals.SIGTRAP: 5>: 'SIGTRAP', <Signals.SIGTERM: 15>: 'SIGTERM', <Signals.SIGSYS: 31>: 'SIGSYS', <Signals.SIGSTOP: 19>: 'SIGSTOP', <Signals.SIGSEGV: 11>: 'SIGSEGV', <Signals.SIGRTMIN: 34>: 'SIGRTMIN', <Signals.SIGRTMAX: 64>: 'SIGRTMAX', <Signals.SIGQUIT: 3>: 'SIGQUIT', <Signals.SIGPWR: 30>: 'SIGPWR', <Signals.SIGPROF: 27>: 'SIGPROF', <Signals.SIGIO: 29>: 'SIGIO', <Signals.SIGPIPE: 13>: 'SIGPIPE', <Signals.SIGKILL: 9>: 'SIGKILL', <Signals.SIGABRT: 6>: 'SIGABRT', <Signals.SIGINT: 2>: 'SIGINT', <Signals.SIGILL: 4>: 'SIGILL', <Signals.SIGHUP: 1>: 'SIGHUP', <Signals.SIGFPE: 8>: 'SIGFPE', <Signals.SIGCONT: 18>: 'SIGCONT', <Signals.SIGCHLD: 17>: 'SIGCHLD', <Signals.SIGBUS: 7>: 'SIGBUS', <Signals.SIGALRM: 14>: 'SIGALRM'}
  • Monkey Patching

    Monkey patching in Python refers to the practice of dynamically modifying or extending code at runtime typically replacing or adding new functionalities to existing modulesclasses or methods without altering their original source code. This technique is often used for quick fixes, debugging or adding temporary features.

    The term “monkey patching” originates from the idea of making changes in a way that is ad-hoc or temporary, akin to how a monkey might patch something up using whatever materials are at hand.

    Steps to Perform Monkey Patching

    Following are the steps that shows how we can perform monkey patching −

    • First to apply a monkey patch we have to import the module or class we want to modify.
    • In the second step we have to define a new function or method with the desired behavior.
    • Replace the original function or method with the new implementation by assigning it to the attribute of the class or module.

    Example of Monkey Patching

    Now let’s understand the Monkey patching with the help of an example −

    Define a Class or Module to Patch

    First we have to define the original class or module that we want to modify. Below is the code −

    # original_module.pyclassMyClass:defsay_hello(self):return"Hello, Welcome to Tutorialspoint!"

    Create a Patching Function or Method

    Next we have to define a function or method that we will use to monkey patch the original class or module. This function will contain the new behavior or functionality we want to add −

    # patch_module.pyfrom original_module import MyClass
    
    # Define a new function to be patcheddefnew_say_hello(self):return"Greetings!"# Monkey patching MyClass with new_say_hello method
    MyClass.say_hello = new_say_hello
    

    Test the Monkey Patch

    Now we can test the patched functionality. Ensure that the patching is done before we create an instance of MyClass with the patched method −

    # test_patch.pyfrom original_module import MyClass
    import patch_module
    
    # Create an instance of MyClass
    obj = MyClass()# Test the patched methodprint(obj.say_hello())# Output: Greetings!

    Drawbacks of Monkey Patching

    Following are the draw backs of monkey patching −

    • Overuse: Excessive monkey patching can lead to code that is hard to understand and maintain. We have to use it judiciously and consider alternative design patterns if possible.
    • Compatibility: Monkey patching may introduce unexpected behavior especially in complex systems or with large code-bases.
  • Mocking and Stubbing

    Python mocking and stubbing are important techniques in unit testing that help to isolate the functionality being tested by replacing real objects or methods with controlled substitutes. In this chapter we are going to understand about Mocking and Stubbing in detail −

    Python Mocking

    Mocking is a testing technique in which mock objects are created to simulate the behavior of real objects.

    This is useful when testing a piece of code that interacts with complex, unpredictable or slow components such as databases, web services or hardware devices.

    The primary purpose of mocking is to isolate the code under test and ensure that its behavior is evaluated independently of its dependencies.

    Key Characteristics of Mocking

    The following are the key characteristics of mocking in python −

    • Behavior Simulation: Mock objects can be programmed to return specific values, raise exceptions or mimic the behavior of real objects under various conditions.
    • Interaction Verification: Mocks can record how they were used by allowing the tester to verify that specific methods were called with the expected arguments.
    • Test Isolation:By replacing real objects with mocks, tests can focus on the logic of the code under test without worrying about the complexities or availability of external dependencies.

    Example of Python Mocking

    Following is the example of the database.get_user method, which is mocked to return a predefined user dictionary. The test can then verify that the method was called with the correct arguments −

    from unittest.mock import Mock
    
    # Create a mock object
    database = Mock()# Simulate a method call
    database.get_user.return_value ={"name":"Prasad","age":30}# Use the mock object
    user = database.get_user("prasad_id")print(user)# Verify the interaction
    database.get_user.assert_called_with("prasad_id")

    Output

    {'name': 'Prasad', 'age': 30}
    

    Python Stubbing

    Stubbing is a related testing technique where certain methods or functions are replaced with “stubs” that return fixed, predetermined responses.

    Stubbing is simpler than mocking because it typically does not involve recording or verifying interactions. Instead, stubbing focuses on providing controlled inputs to the code under test by ensuring consistent and repeatable results.

    Key Characteristics of Stubbing

    The following are the key characteristics of Stubbing in python −

    • Fixed Responses: Stubs return specific, predefined values or responses regardless of how they are called.
    • Simplified Dependencies: By replacing complex methods with stubs, tests can avoid the need to set up or manage intricate dependencies.
    • Focus on Inputs: Stubbing emphasizes providing known inputs to the code under test by allowing the tester to focus on the logic and output of the tested code.

    Example of Python Stubbing

    Following is the example of the get_user_from_db function, which is stubbed to always return a predefined user dictionary. The test does not need to interact with a real database for simplifying the setup and ensuring consistent results −

    from unittest.mock import patch
    
    # Define the function to be stubbeddefget_user_from_db(user_id):# Simulate a complex database operationpass# Test the function with a stubwith patch('__main__.get_user_from_db', return_value={"name":"Prasad","age":25}):
       user = get_user_from_db("prasad_id")print(user)

    Output

    {'name': 'Prasad', 'age': 25}
    

    Python Mocking Vs. Stubbing

    The comparison of the Mocking and Stubbing key features, purposes and use cases gives the clarity on when to use each method. By exploring these distinctions, developers can create more effective and maintainable tests which ultimately leads to higher quality software.

    The following table shows the key difference between mocking and stubbing based on the different criteria −

    CriteriaMockingStubbing
    PurposeSimulate the behavior of real objectsProvide fixed, predetermined responses
    Interaction VerificationCan verify method calls and argumentsTypically does not verify interactions
    ComplexityMore complex; can simulate various behaviorsSimpler; focuses on providing controlled inputs
    Use CaseIsolate and test code with complex dependenciesSimplify tests by providing known responses
    Recording BehaviorRecords how methods were calledDoes not record interactions
    State ManagementCan maintain state across callsUsually stateless; returns fixed output
    Framework SupportPrimarily uses unittest.mock with features like Mock and MagicMockUses unittest.mock’s patch for simple replacements
    FlexibilityHighly flexible; can simulate exceptions and side effectsLimited flexibility; focused on return values
  • Metaprogramming with Metaclasses

    In Python, Metaprogramming refers to the practice of writing code that has knowledge of itself and can be manipulated. The metaclasses are a powerful tool for metaprogramming in Python, allowing you to customize how classes are created and behave. Using metaclasses, you can create more flexible and efficient programs through dynamic code generation and reflection.

    Metaprogramming in Python involves techniques such as decorators and metaclasses. In this tutorial, you will learn about metaprogramming with metaclasses by exploring dynamic code generation and reflection.

    Defining Metaclasses

    Metaprogramming with metaclasses in Python offer advanced features of enabling advanced capabilities to your program. One such feature is the __prepare__() method, which allows customization of the namespace where a class body will be executed.

    This method is called before any class body code is executed, providing a way to initialize the class namespace with additional attributes or methods. The __prepare__() method should be implemented as a classmethod.

    Example

    Here is an example of creating metaclass with advanced features using the __prepare__() method.

    classMyMetaClass(type):@classmethoddef__prepare__(cls, name, bases,**kwargs):print(f'Preparing namespace for {name}')# Customize the namespace preparation here
    
      custom_namespace =super().__prepare__(name, bases,**kwargs)
      custom_namespace['CONSTANT_VALUE']=100return custom_namespace
    # Define a class using the custom metaclassclassMyClass(metaclass=MyMetaClass):def__init__(self, value):
      self.value = value
    
    defdisplay(self):print(f"Value: {self.value}, Constant: {self.__class__.CONSTANT_VALUE}")# Instantiate the class obj = MyClass(42) obj.display()

    Output

    While executing above code, you will get the following results −

    Preparing namespace for MyClass
    Value: 42, Constant: 100
    

    Dynamic Code Generation with Metaclasses

    Metaprogramming with metaclasses enables the creation or modification of code during runtime.

    Example

    This example demonstrates how metaclasses in Python metaprogramming can be used for dynamic code generation.

    classMyMeta(type):def__new__(cls, name, bases, attrs):print(f"Defining class: {name}")# Dynamic attribute to the class
    
      attrs['dynamic_attribute']='Added dynamically'# Dynamic method to the classdefdynamic_method(self):returnf"This is a dynamically added method for {name}"
        
      attrs['dynamic_method']= dynamic_method
        
      returnsuper().__new__(cls, name, bases, attrs)# Define a class using the metaclassclassMyClass(metaclass=MyMeta):pass
    obj = MyClass()print(obj.dynamic_attribute)print(obj.dynamic_method())

    Output

    On executing above code, you will get the following results −

    Defining class: MyClass
    Added dynamically
    This is a dynamically added method for MyClass
    

    Reflection and Metaprogramming

    Metaprogramming with metaclasses often involves reflection, allowing for introspection and modification of class attributes and methods at runtime.

    Example

    In this example, the MyMeta metaclass inspects and prints the attributes of the MyClass during its creation, demonstrating how metaclasses can introspect and modify class definitions dynamically.

    classMyMeta(type):def__new__(cls, name, bases, dct):# Inspect class attributes and print themprint(f"Class attributes for {name}: {dct}")returnsuper().__new__(cls, name, bases, dct)classMyClass(metaclass=MyMeta):
       data ="example"

    Output

    On executing above code, you will get the following results −

    Class attributes for MyClass: {'__module__': '__main__', '__qualname__': 'MyClass', 'data': 'example'}
    
  •  Metaclasses

    Metaclasses are a powerful feature in Python that allow you to customize class creation. By using metaclasses, you can add specific behaviors, attributes, and methods to classes, and allowing you to create more flexible, efficient programs. This classes provides the ability to work with metaprogramming in Python.

    Metaclasses are an OOP concept present in all python code by default. Python provides the functionality to create custom metaclasses by using the keyword type. Type is a metaclass whose instances are classes. Any class created in python is an instance of type metaclass.

    Creating Metaclasses in Python

    A metaclass is a class of a class that defines how a class behaves. Every class in Python is an instance of its metaclass. By default, Python uses type() function to construct the metaclasses. However, you can define your own metaclass to customize class creation and behavior.

    When defining a class, if no base classes or metaclass are explicitly specified, then Python uses type() to construct the class. Then its body is executed in a new namespace, resulting class name is locally linked to the output of type(name, bases, namespace).

    Example

    Let’s observe the result of creating a class object without specifying specific bases or a metaclass

    classDemo:pass
       
    obj = Demo()print(obj)

    Output

    On executing the above program, you will get the following results −

    <__main__.Demo object at 0x7fe78f43fe80>
    

    This example demonstrates the basics of metaprogramming in Python using metaclasses. The above output indicates that obj is an instance of the Demo class, residing in memory location 0x7fe78f43fe80. This is the default behavior of the Python metaclass, allowing us to easily inspect the details of the class.

    Creating Metaclasses Dynamically

    The type() function in Python can be used to create classes metaclasses dynamically.

    Example

    In this example, DemoClass will created using type() function, and an instance of this class is also created and displayed.

    # Creating a class dynamically using type()
    DemoClass =type('DemoClass',(),{})
    obj = DemoClass()print(obj)

    Output

    Upon executing the above program, you will get the following results −

    <__main__.DemoClass object at 0x7f9ff6af3ee0>
    

    Example

    Here is another example of creating a Metaclass with inheritance which can be done by inheriting one from another class using type() function.

    classDemo:pass
       
    Demo2 =type('Demo2',(Demo,),dict(attribute=10))
    obj = Demo2()print(obj.attribute)print(obj.__class__)print(obj.__class__.__bases__)

    Output

    Following is the output −

    10
    <class '__main__.Demo2'>
    (<class '__main__.Demo'>,)
    

    Customizing Metaclass Creation

    In Python, you can customize how classes are created and initialized by defining your own metaclass. This customization is useful for various metaprogramming tasks, such as adding specific behavior to all instances of a class or enforcing certain patterns across multiple classes.

    Customizing the classes can be done by overriding methods in the metaclass, specifically __new__ and __init__.

    Example

    Let’s see the example of demonstrating how we can customize class creation using the __new__ method of a metaclass in python.

    # Define a custom metaclassclassMyMetaClass(type):def__new__(cls, name, bases, dct):
    
      dct['version']=1.0# Modify the class name
      name ='Custom'+ name
        
      returnsuper().__new__(cls, name, bases, dct)# MetaClass acts as a template for the custom metaclassclassDemo(metaclass=MyMetaClass):pass# Instantiate the class
    obj = Demo()# Print the class name and version attributeprint("Class Name:",type(obj).__name__)print("Version:", obj.version)

    Output

    While executing above code, you will get the following results −

    Class Name: CustomDemo
    Version: 1.0
    

    Example

    Here is another example that demonstrates how to customize the metaclass using the __init__ in Python.

    # Define a custom metaclassclass2yCreating MetaClass(type):def__init__(cls, name, bases, dct):print('Initializing class', name)# Add a class-level attribute
    
      cls.version=10super().__init__(name, bases, dct)# Define a class using the custom metaclassclassMyClass(metaclass=MyMetaClass):def__init__(self, value):
      self.value = value
    
    defdisplay(self):print(f"Value: {self.value}, Version: {self.__class__.version}")# Instantiate the class and demonstrate its usage obj = MyClass(42) obj.display()

    Output

    While executing above code, you will get the following results −

    Initializing class MyClass
    Value: 42, Version: 10
  •  Memory Management

    In Python, memory management is automatic, it involves handling a private heap that contains all Python objects and data structures. The Python memory manager internally ensures the efficient allocation and deallocation of this memory. This tutorial will explore Python’s memory management mechanisms, including garbage collection, reference counting, and how variables are stored on the stack and heap.

    Memory Management Components

    Python’s memory management components provides efficient and effective utilization of memory resources throughout the execution of Python programs. Python has three memory management components −

    • Private Heap: Acts as the main storage for all Python objects and data. It is managed internally by the Python memory manager.
    • Raw Memory Allocator: This low-level component directly interacts with the operating system to reserve memory space in Python’s private heap. It ensures there’s enough room for Python’s data structures and objects.
    • Object-Specific Allocators: On top of the raw memory allocator, several object-specific allocators manage memory for different types of objects, such as integers, strings, tuples, and dictionaries.

    Memory Allocation in Python

    Python manages memory allocation in two primary ways − Stack and Heap.

    Stack − Static Memory Allocation

    In static memory allocation, memory is allocated at compile time and stored in the stack. This is typical for function call stacks and variable references. The stack is a region of memory used for storing local variables and function call information. It operates on a Last-In-First-Out (LIFO) basis, where the most recently added item is the first to be removed.

    The stack is generally used for variables of primitive data types, such as numbers, booleans, and characters. These variables have a fixed memory size, which is known at compile-time.

    Example

    Let us look at an example to illustrate how variables of primitive types are stored on the stack. In the above example, variables named x, y, and z are local variables within the function named example_function(). They are stored on the stack, and when the function execution completes, they are automatically removed from the stack.

    defmy_function():
       x =5
       y =True
       z ='Hello'return x, y, z
    
    print(my_function())print(x, y, z)

    On executing the above program, you will get the following output −

    (5, True, 'Hello')
    Traceback (most recent call last):
      File "/home/cg/root/71937/main.py", line 8, in <module>
    
    print(x, y, z)
    NameError: name 'x' is not defined

    Heap − Dynamic Memory Allocation

    Dynamic memory allocation occurs at runtime for objects and data structures of non-primitive types. The actual data of these objects is stored in the heap, while references to them are stored on the stack.

    Example

    Let’s observe an example for creating a list dynamically allocates memory in the heap.

    a =[0]*10print(a)

    Output

    On executing the above program, you will get the following results −

    [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    

    Garbage Collection in Python

    Garbage Collection in Python is the process of automatically freeing up memory that is no longer in use by objects, making it available for other objects. Pythons garbage collector runs during program execution and activates when an object’s reference count drops to zero.

    Reference Counting

    Python’s primary garbage collection mechanism is reference counting. Every object in Python maintains a reference count that tracks how many aliases (or references) point to it. When an object’s reference count drops to zero, the garbage collector deallocates the object.

    Working of the reference counting as follows −

    • Increasing Reference Count− It happens when a new reference to an object is created, the reference count increases.
    • Decreasing Reference Count− When a reference to an object is removed or goes out of scope, the reference count decreases.

    Example

    Here is an example that demonstrates working of reference counting in Python.

    import sys
    
    # Create a string object
    name ="Tutorialspoint"print("Initial reference count:", sys.getrefcount(name))# Assign the same string to another variable
    other_name ="Tutorialspoint"print("Reference count after assignment:", sys.getrefcount(name))# Concatenate the string with another string
    string_sum = name +' Python'print("Reference count after concatenation:", sys.getrefcount(name))# Put the name inside a list multiple times
    list_of_names =[name, name, name]print("Reference count after creating a list with 'name' 3 times:", sys.getrefcount(name))# Deleting one more reference to 'name'del other_name
    print("Reference count after deleting 'other_name':", sys.getrefcount(name))# Deleting the list referencedel list_of_names
    print("Reference count after deleting the list:", sys.getrefcount(name))

    Output

    On executing the above program, you will get the following results −

    Initial reference count: 4
    Reference count after assignment: 5
    Reference count after concatenation: 5
    Reference count after creating a list with 'name' 3 times: 8
    Reference count after deleting 'other_name': 7
    Reference count after deleting the list: 4
    
  • Object Internals

    The internals of Python objects provides deeper insights into how Python manages and manipulates data. This knowledge is essential for writing efficient, optimized code and for effective debugging.

    Whether we’re handling immutable or mutable objects by managing memory with reference counting and garbage collection or leveraging special methods and slots, grasping these concepts is fundamental to mastering Python programming.

    Understanding Python’s object internals is crucial for optimizing code and debugging. Following is an overview of the key aspects of Python object internals −

    Object Structure

    In Python every object is a complex data structure that encapsulates various pieces of information. Understanding the object structure helps developers to grasp how Python manages memory and handles data.

    Each python object mainly consists of two parts as mentioned below −

    • Object Header: This is a crucial part of every Python object that contains essential information for the Python interpreter to manage the object effectively. It typically consists of two main components namely Reference count and Type Pointer.
    • Object Data: This data is the actual data contained within the object which can differ based on the object’s type. For example an integer contains its numeric value while a list contains references to its elements.
    • Object Identity
      Object Identity is the identity of an object which is an unique integer that represents its memory address. It remains constant during the object’s lifetime. Every object in Python has a unique identifier obtained using the id() function.
      Example
      Following is the example code of getting the Object Identity −


      a = “Tutorialspoint” print(id(a)) # Example of getting the id of an string object
      On executing the above code we will get the following output −
      2366993878000
      Note: The memory address will change on every execution of the code.
      Object Type
      Object Type is the type of an object defines the operations that can be performed on it. For example integers, strings and lists have distinct types. It is defined by its class and can be accessed using the type() function.
      Example
      Here is the example of it −

      a = “Tutorialspoint” print(type(a))
      On executing the above code we will get the following output −
      <class ‘str’>
      Object Value
      Object Value of an object is the actual data it holds. This can be a primitive value like an integer or string, or it can be more complex data structures like listsor dictionaries.
      Example
      Following is the example of the object value −


      b = “Welcome to Tutorialspoint” print(b)
      On executing the above code we will get the following output −
      Welcome to Tutorialspoint
      Memory Management
      Memory management in Python is a critical aspect of the language’s design by ensuring efficient use of resources while handling object lifetimes and garbage collection. Here are the key components of memory management in Python −
      Reference Counting: Python uses reference counting to manage memory. Each object keeps track of how many references point to it. When this count drops to zero then the memory can be freed.
      Garbage Collection: In addition to reference counting the Python employs a garbage collector to identify and clean up reference cycles.
      Example
      Following is the example of the getting the reference counting in memory management −

      import sys c = [1, 2, 3] print(sys.getrefcount(c)) # Shows the reference count
      On executing the above code we will get the following output −
      2
      Attributes and Methods
      Python objects can have attributes and methods which are accessed using dot notation. In which Attributes store data while methods define the behavior.
      Example


      class MyClass: def __init__(self, value): self.value = value def display(self): print(self.value) obj = MyClass(10) obj.display()
      On executing the above code we will get the following output −
      10
      Finally, understanding Python’s object internals helps optimize performance and debug effectively. By grasping how objects are structured and managed in memory where developers can make informed decisions when writing Python code
  • Higher Order Functions

    Higher-order functions in Python allows you to manipulate functions for increasing the flexibility and re-usability of your code. You can create higher-order functions using nested scopes or callable objects.

    Additionally, the functools module provides utilities for working with higher-order functions, making it easier to create decorators and other function-manipulating constructs. This tutorial will explore the concept of higher-order functions in Python and demonstrate how to create them.

    What is a Higher-Order Function?

    A higher-order function is a function that either, takes one or more functions as arguments or returns a function as its result. Below you can observe the some of the properties of the higher-order function in Python −

    • A function can be stored in a variable.
    • A function can be passed as a parameter to another function.
    • A high order functions can be stored in the form of lists, hash tables, etc.
    • Function can be returned from a function.

    To create higher-order function in Python you can use nested scopes or callable objects. Below we will discuss about them briefly.

    Creating Higher Order Function with Nested Scopes

    One way to defining a higher-order function in Python is by using nested scopes. This involves defining a function within another function and returns the inner function.

    Example

    Let’s observe following example for creating a higher order function in Python. In this example, the multiplier function takes one argument, a, and returns another function multiply, which calculates the value a * b

    defmultiplier(a):# Nested function with second number   defmultiply(b):# Multiplication of two numbers  return a * b 
       return multiply   
    
    # Assigning nested multiply function to a variable  
    multiply_second_number = multiplier(5)# Using variable as high order function  
    Result = multiply_second_number(10)# Printing result  print("Multiplication of Two numbers is: ", Result)

    Output

    On executing the above program, you will get the following results −

    Multiplication of Two numbers is:  50
    

    Creating Higher-Order Functions with Callable Objects

    Another approach to create higher-order functions is by using callable objects. This involves defining a class with a __call__ method.

    Example

    Here is the another approach to creating higher-order functions is using callable objects.

    classMultiplier:def__init__(self, factor):
    
      self.factor = factor
    def__call__(self, x):return self.factor * x # Create an instance of the Multiplier class multiply_second_number = Multiplier(2)# Call the Multiplier object to computes factor * x Result = multiply_second_number(100)# Printing result print("Multiplication of Two numbers is: ", Result)

    Output

    On executing the above program, you will get the following results −

    Multiplication of Two numbers is:  200
    

    Higher-order functions with the ‘functools’ Module

    The functools module provides higher-order functions that act on or return other functions. Any callable object can be treated as a function for the purposes of this module.

    Working with Higher-order functions using the wraps()

    In this example, my_decorator is a higher-order function that modifies the behavior of invite function using the functools.wraps() function.

    import functools
    
    defmy_decorator(f):@functools.wraps(f)defwrapper(*args,**kwargs):print("Calling", f.__name__)return f(*args,**kwargs)return wrapper
    
    @my_decoratordefinvite(name):print(f"Welcome to, {name}!")
    
    invite("Tutorialspoint")

    Output

    On executing the above program, you will get the following results −

    Calling invite
    Welcome to, Tutorialspoint!
    

    Working with Higher-order functions using the partial()

    The partial() function of the functools module is used to create a callable ‘partial’ object. This object itself behaves like a function. The partial() function receives another function as argument and freezes some portion of a functions arguments resulting in a new object with a simplified signature.

    Example

    In following example, a user defined function myfunction() is used as argument to a partial function by setting default value on one of the arguments of original function.

    import functools
    defmyfunction(a,b):return a*b
    
    partfunction = functools.partial(myfunction,b =10)print(partfunction(10))

    Output

    On executing the above program, you will get the following results −

    100
    

    Working with Higher-order functions using the reduce()

    Similar to the above approach the functools module provides the reduce()function, that receives two arguments, a function and an iterable. And, it returns a single value. The argument function is applied cumulatively two arguments in the list from left to right. Result of the function in first call becomes first argument and third item in list becomes second. This is repeated till list is exhausted.

    Example

    import functools
    defmult(x,y):return x*y
    
    # Define a number to calculate factorial
    n =4
    num = functools.reduce(mult,range(1, n+1))print(f'Factorial of {n}: ',num)

    Output

    On executing the above program, you will get the following results −

    Factorial of 4:  24
    
  • Custom Exceptions

    What are Custom Exceptions in Python?

    Python custom exceptions are user-defined error classes that extend the base Exception class. Developers can define and handle specific error conditions that are unique to their application. Developers can improve their code by creating custom exceptions. This allows for more meaningful error messages and facilitates the debugging process by indicating what kind of error occurred and where it originated.

    To define a unique exception we have to typically create a new class that takes its name from Python’s built-in Exception class or one of its subclasses. A corresponding except block can be used to raise this custom class and catch it.

    Developers can control the flow of the program when specific errors occur and take appropriate actions such as logging the error, retrying operations or gracefully shutting down the application. Custom exceptions can carry additional context or data about the error by overriding the __init__ method and storing extra information as instance attributes.

    Using custom exceptions improves the clarity of error handling in complex programs. It helps to distinguish between different types of errors that may require different handling strategies. For example when a file parsing library might define exceptions like FileFormatError, MissingFieldError or InvalidFieldError to handle various issues that can arise during file processing. This level of granularity allows the client code to catch and address specific issues more effectively by improving the robustness and user experience of the software. Python’s custom exceptions are a great tool for handling errors and writing better with more expressive code.

    Why to Use Custom Exceptions?

    Custom exceptions in Python offer several advantages which enhances the functionality, readability and maintainability of our code. Here are the key reasons for using custom exceptions −

    • Specificity: Custom exceptions allow us to represent specific error conditions that are unique to our application.
    • Clarity: They make the code more understandable by clearly describing the nature of the errors.
    • Granularity: Custom exceptions allow for more precise error handling.
    • Consistency: They help to maintain a consistent error-handling strategy across the codebase.

    Creating Custom Exceptions

    Creating custom exceptions in Python involves defining new exception classes that extend from Python’s built-in Exception class or any of its subclasses. This allows developers to create specialized error types that cater to specific scenarios within their applications. Here’s how we can create and use custom exceptions effectively −

    Define the Custom Exception Class

    We can start creating the custom exceptions by defining a new class that inherits from Exception or another exception class such as RuntimeError, ValueError, etc. depending on the nature of the error.

    Following is the example of defining the custom exception class. In this example CustomError is a custom exception class that inherits from Exception. The __init__ method initializes the exception with an optional error message −

    classCustomError(Exception):def__init__(self, message):super().__init__(message)
    
      self.message = message

    Raise the Custom Exception

    To raise the custom exception we can use the raise statement followed by an instance of our custom exception class. Optionally we can pass a message to provide context about the error.

    In this function process_data() the CustomError exception is raised when the data parameter is empty by indicating a specific error condition.

    defprocess_data(data):ifnot data:raise CustomError("Empty data provided")# Processing logic here

    Handle the Custom Exception

    To handle the custom exception we have to use a try-except block. Catch the custom exception class in the except block and handle the error as needed.

    Here in the below code if process_data([]) raises a CustomError then the except block catches it and we can print the error message (e.message) or perform other error-handling tasks.

    try:
       process_data([])except CustomError as e:print(f"Custom Error occurred: {e.message}")# Additional error handling logic

    Example of Custom Exception

    Following is the basic example of custom exception handling in Python. In this example we define a custom exception by subclassing the built-in Exception class and use a try-except block to handle the custom exception −

    # Define a custom exceptionclassCustomError(Exception):def__init__(self, message):
    
      self.message = message
      super().__init__(self.message)# Function that raises the custom exceptiondefcheck_value(value):if value &lt;0:raise CustomError("Value cannot be negative!")else:returnf"Value is {value}"# Using the function with exception handlingtry:
    result = check_value(-5)print(result)except CustomError as e:print(f"Caught an exception: {e.message}")

    Output

    On executing the above code we will get the following output −

    Caught an exception: Value cannot be negative!