Git Integration: When To Choose Git

Overview

This guide is designed to help users determine whether the Git integration feature for Matillion ETL is suitable for their use case.

Git is a distributed version control system that lets software development teams track a project's changes (both small and large) with speed and efficiency.

When To Choose Git

Note: Git is an "enterprise" feature within Matillion ETL. To learn more about Matillion's Enterprise Mode and how to upgrade your licence, please click here.

Matillion ETL lets users convert their Matillion Project into a Git project, and use Matillion's Versions management system as a network of independent working areas, centralized via the Git feature, where users can perform actions including Commit, Create Branch, Configure Remote Repository, Fetch, and Push. Each Matillion Version within a Git project points at a specific Git commit.

Choosing Matillion ETL's Git support enables users to:

• Perform branch-focused software development with Matillion jobs - typically adhering to common software engineering workflows.
• Move Matillion ETL jobs between projects - for example, between Development, Testing, and Production projects.
• Commit code to a remote Git repository, such as GitHub, AWS CodeCommit, or Bitbucket.

Suggested Requirements

If the below-listed situations match a user experience of Matillion ETL, we believe the Git feature may be appropriate:

• Where the user's organization already mandates the use of Git in software engineering.
• Where an organization's Matillion ETL user(s) already have experience with a version control system such as Git.
• Where an organization wishes to use an off-site or remote Git repository to externally backup their code.

Considerations

Git is a flexible and complex tool, and teams opting to use Matillion ETL's Git support should establish a workflow that satisfies their specific use case. We have designed an example of a typical workflow for using Git in Matillion ETL.

Users should also ensure they have access to an external repository, as mentioned above, for example, GitHub. Users should also be aware that actions involving managing the workflow, managing the external repository, and managing user permissions will be required.

Note: Matillion ETL's Git support only allows storage of versionable items, not environmental items.

Versionable items (can be stored in Git):

• Orchestration Jobs
• Transformation Jobs

Environmental items (cannot be stored in Git):

• Matillion Environments
• Environment Variables
• Matillion project configuration features, such as CDC or SQS

Alternatives to using Git Support in Matillion ETL

For many users, integrating Git into their Matillion ETL projects may be offset by simpler alternatives.

Matillion has a built-in versioning system, which enables users to conduct basic version control such as tagging releases and tracking changes. For more information, read Manage Versions.

Matillion also has a server migration tool, which lets users move their ETL jobs and other collateral, such as OAuths, between Matillion projects without having to adopt an external Git service. Read Migrate for more information.

Users can interact with the Matillion API to automate moving jobs between projects, and perform additional environment configuration. Users may find this feature suitable if:

• They have a pre-existing external scheduler for their jobs.
• They need to integrate the deployment of Matillion ETL within a wider automated progress.
• The user wishes to design a bespoke process. Please see our Matillion API documentation for more information.