Metadata-Version: 2.1
Name: pipelayer
Version: 0.2.0
Summary: A lightweight pipeline framework
Home-page: https://github.com/greater-than/PipeLayer
Author: greaterThan, LLC
Author-email: info@greaterthan.solutions
License: UNKNOWN
Description: # PipeLayer
        PipeLayer is a lightweight Python pipeline framework. Define a series of filters, and chain them together to create modular applications.
        <br>
        
        ### Table of Contents
        
        * [Installation](#installation)
        * [Getting Started](#getting-started)
        * [The Framework](#the-framework)
        * _[VERSION HISTORY](HISTORY.md)_
        * _[LICENSE](LICENSE.txt)_
        <br><br>
        
        ## Installation
        
        From the command line:
        ```sh
        pip install pipelayer
        ```
        
        ## Getting Started
        
        ### Step 1: Application Settings
        Create a class called AppSettings that inherits from [`pipelayer.Settings`](Settings):
        
        `app_settings.py`
        ```python
        from pipelayer import Settings
        
        
        class AppSettings(Settings):
            """
            Complete this by adding constants, key/value data from AWS Parameter Store, etc
        
            The pipelayer.Settings class inherits from pydantic.BaseModel, so fields must be typed appropriately
            """
            ...
        ```
        NOTE: You do not have to use the Settings base class in your application. Any class can be provided.
        
        ### Step 2: Application Context
        Create a class called AppContext that inherits from the `pipelayer.Context` class:
        
        `app_context.py`
        ```python
        from logging import Logger
        
        from app_settings import AppSettings
        from pipelayer import Context
        
        
        class AppContext(Context):
            def __init__(self, settings: AppSettings, log: Logger = Nones):
                self.__settings = settings
                self.__log = log
        
            @property
            def settings(self) -> AppSettings:
                return self.__settings
        
            @property
            def log(self) -> Logger:
                return self.__log
        ```
        
        ### Step 3: Create Pipeline Filters
        Filters can be defined in the following ways:
        * Classes that inherits from `pipelayer.Filter` and implement the `run` method
        * Functions (instance/class/static/module) that have the following signature `func(context: Any, data: Any)`
        * Anonymous functions (lambda) with two arguments that follow the same pattern for regular functions: `my_func = lambda context, data: data`
        
        `hello_world_filters.py`
        ```python
        from pipelayer import Filter
        
        
        class HelloFilter(Filter):
            def run(self, context) -> str:
                return "Hello"
        
        
        class WorldFilter(Filter):
            def run(self, context, data) -> str:
                return f"{data},  World!"
        ```
        
        `functions.py`
        ```python
        def create_message_dict(context, data: str) -> dict:
            return {"message": data}
        ```
        
        ### Step 4: Create a Pipeline
        Create a module to run the pipeline:
        
        `app.py`
        ```python
        from logging import Logger
        from pipelayer import Pipeline
        
        from app_context import AppContext
        from app_settings import AppSettings
        from functions import create_message
        from hello_world_filters import HelloFilter, WorldFilter
        
        app_settings = AppSettings()
        app_context = AppContext(app_settings, Logger("Logger"))
        hello_world_pipeline = Pipeline.create(app_context, "Hello World Pipeline")
        
        output = hello_world_pipeline.run([
            HelloFilter,                           # pipeline.Filter type
            WorldFilter(),                         # pipeline.Filter instance
            create_message_dict                    # function type
            lambda context, data: json.dumps(data) # anonymous function
        ])
        
        # output = '{"message": "Hello, World!"}'
        
        print(f"Pipeline Output: {output}")
        print(hello_world_pipeline.manifest.__dict__)
        
        ```
        
        ### Step 5: Run the Pipeline
        from the command line:
        ```sh
        run app.py
        ```
        ---
        
        ## The Framework
        * [Pipeline](#pipelayer.pipeline)
        * [Filter](#pipelayer.filter)
        * [Context](#pipelayer.context)
        * [Settings](#pipelayer.settings)
        * [Manifest](#pipelayer.manifest)
        * [Utilities](#utilities)
        <br><br>
        
        
        ### __`pipelayer.Pipeline`__
        
        ***Properties***
        
        __`name`__ (str)<br>
        An optional name. It's used in the `Manifest`.
        
        __`context`__ (Context)<br>
        An instance of `pipelayer.Context`.
        
        __`manifest`__ (Manifest)<br>
        An instance of `pipelayer.Manifest` that is created at runtime.
        
        ***Methods***
        
        __`create(context: Union[Context, Any], name: str = "") -> Pipeline`__<br>
        The factory method to create a pipeline
        
        __`run(filters: List[Filter], data: Any = None) -> Any`__<br>
        The pipeline runner that iterates through the `filters` and pipes filter output to the next filter.
        <br><br>
        
        
        ### __`pipelayer.Filter`__
        A functional unit that implements the `run` method, and the optional `pre_process` and `post_process` methods.
        
        ***Properties***
        
        __`name`__ (str)<br>
        Optional. Used by the [`Manifest`](#pipelayer.manifest)
        
        __`pre_process`__ (callable)<br>
        Optional.
        
        __`post_process`__ (callable)<br>
        Optional.
        
        ***Methods***
        
        __`run(context: Union[Context, Any], data: Any) -> Any`__<br>
        The type of the `data` argument in the abstract class is `Any`, but you can use the correct type for the data when implementing method. The same is true for the return type.
        <br><br>
        
        
        ### __`pipelayer.Context`__
        The Context object is used to pass application-level information to each filter.
        
        ***Properties***
        
        __[`settings`](#settings)__ (Any)<br>
        An optional abstract property for storing application-level data.
        
        __`log`__ (Any)<br>
        An optional abstract property for a common logger that can be used within pipeline filters.
        
        __[`manifest`](#manifest)__ (Manifest)<br>
        A model that keeps a record of start/stop times for a pipeline, as well as each filter.
        <br><br>
        
        
        ### __`pipelayer.Settings`__
        The settings class is an optional base class for applications settings that inherits from `pydantic.BaseModel`.
        <br><br>
        
        
        ### __`pipelayer.Manifest`__
        The Manifest keeps a record of [`Pipeline`](#pipeline) and [`Filter`](#filter) activity.
        
        
        ### Utilities
        
        ### __`pipelayer.util.render_manifest(manifest: Manifest, indent: int = 2) -> str`__
        Static function that renders formatted JSON data
        
        
Platform: UNKNOWN
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.8
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.7
Description-Content-Type: text/markdown
