This is however considered a bad practice for Top Level code, also using DB access for callbacks (TODO discuss it) is not recommended practice. Essentially users can access Airflow Variables, Connections and other DB models while the DAG code is parsed or when callbacks are executed. In the future, the JWT token can be used to carry additional (signed) payload which will be used to selectively authorize the caller to specific entities (connections, variables, specific tasks or dags only etc.) No access to Metadata DB for “User code” in DagProcessor and TriggereĬurrently, SQLAlchemy Models of Airflow are available also for the top-level user code from DAGs and callbacks. While this AIP does not address it, it makes it possible. Later phase of “Umbrella” AIP-1 implementation (not implemented in this AIP) is to implement task-based authorization (each task should have access to a subset of operations depending on dag_id, task id, origin, etc.). As the next step (implementing POC) we will determine if we should generate the endpoints from the code via completely custom code, or whether we will be able to leverage existing libraries and tools (for example Fast API). The Internal endpoints do not need to be published in the API specification so we plan to generate the APIs based on the code (decorated functions). The Internal API calls are “RPC”-style APIs rather than CRUD APIS so they do not map into the REST semantics. This component can be run in a separate “security zone”, allowing to separate access to the “Internal API” on a physical/networking level We also propose to implement a separate, optional “airflow internal-api” component, that could expose only the APIs needed by Workers, DAGProcessor and Triggerer. The “Internal API” optionally can also be exposed by a component separate from the current webserver.Physical separation of the components exposing both APIs:.This will be done using the same RBAC controls that we use in current REST API implementation. We will make an inventory and make a decision on a “default” permissions for REST APIs during the implementation. Selected (possibly even all) other API methods from existing REST API that should be exposed to the Workers, Dag Processor and Triggerer. This “Internal API Role” will have only permissions that will allow it to execute the Internal APIs (RPC-styled) that are needed to provide user-code with current functionality (Connections/Variables etc.) in a transparent way.Dag Processor and Triggerer will have a secret that will be available only to them, not to the user code they run (see below the section on “Securing access to authorization token in Dag Processor and Triggerer).The user code in the Worker should not have access to any long term secret - they should only use limited-time valid JWT Token.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |