API v1 - Shared Jobs
    • Dark
      Light

    API v1 - Shared Jobs

    • Dark
      Light

    Article Summary

    Overview

    The Shared Job endpoint allows users to explore what shared jobs are available on the instance as well as importing, exporting and deleting them.
    You can manage your shared jobs in your Matillion ETL instance by clicking the Project Menu, then Manage Shared Jobs.

    Note
    • This document is part of a series on shared jobs and the Matillion ETL API - v1. For related articles, refer to Shared jobs, Manage shared jobs.
    • Users wanting to make use of this endpoint will require your Matillion ETL instance URL, the username and password with appropriate permissions, and an account with the API Role and Shared Jobs permissions.
    • Users responsible for experimenting with Matillion ETL API services require access to the Matillion ETL instance and should know how to make REST API calls either employing a REST API GUI client such as Postman, or employing a command-line interface like cURL.

    URL parameters and descriptions

    Below is the list of endpoint parameters and descriptions for the shared job API:

    Parameter NameDescription
    instance_addressThe server IP address or domain name.
    shared_jobsThe shared jobs in your Matillion ETL instance.
    packageNameThe name of the package within the shared jobs.
    jobNameThe name of the job within the package.
    revisionThe revision listed within the job package of shared jobs.
    exportExports the metadata of the resource.
    importImports the metadata to the resource.
    deleteDeletes the selected resource.

    Shared job API endpoints

    Base URL

    http(s)://{instanceAddress}/rest/v1/sharedjob
    

    API endpoints and function

    The shared job endpoint API is available on standard REST-based APIs that uses HTTP or HTTPS request to GET, POST, and DELETE data. The shared job endpoint API service is accessed through the Uniform Resource Identifier (URI). The available API endpoints are listed below:

    Note

    A "package" represents a folder that can house both other packages and shared jobs. A "package name" is the concatenated path of packages that leads to the shared job. The "package name" shouldn't be mistaken for the literal name of a single package. Nested packages are denoted by a period/full stop. For example, a package name could be packageA.packageB.packageC. If a single package's name includes a space, replace the space with %20 in your API request. Read Shared job configuration for more information.

    MethodPathURIFunction
    GETexporthttp://{instanceAddress}/rest/v1/sharedjob/exportTo export the metadata of the shared jobs within the instance.
    POSTimporthttp://{instanceAddress}/rest/v1/sharedjob/importTo import the data of the shared jobs within the instance.
    GETpackagehttp://{instanceAddress}/rest/v1/sharedjob/packageTo get the package available within the shared job of the instance.

    {packageName}

    MethodPathURIFunction
    GETpackageName/exporthttp://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/exportTo export the specific package from the shared job within the instance.
    GETpackageName/jobhttp://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/jobTo get the list of Jobs available within the current package of the instance.
    POSTpackageName/deletehttp://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/deleteTo delete the selected package from the list using HTTP POST request.
    DELETEpackageNamehttp://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}Delete all jobs within a package.

    {jobName}

    MethodPathURIFunction
    GETjobName/exporthttp://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/exportTo export the specific job available within the package.
    GETjobName/revisionhttp://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/revisionTo get the revisions available within the selected job.
    POSTjobName/deletehttp://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/deleteDelete all revisions of a single job.
    DELETEjobNamehttp://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}Delete all revisions of a single job.

    {revisionId}

    MethodPathURIFunction
    GETrevisionId/exporthttp://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/revision/id/{revisionId}/exportTo export the specific job available within the package.
    POSTrevisionIdhttp://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}revision/id/{revisionId}/Delete a single revision.

    Graphical representation

    To illustrate the API v1 shared job, endpoints and methods, below is the graphical flow of the /sharedjob endpoint showing the PATH, GET, POST, and DELETE options.

    shared job endpoint Flow


    Endpoints and server response

    This section describes the shared job API endpoints with examples. These APIs offer REST-based web service, offering ease of use and a flexible choice of programming language. These APIs can be used to access and analyze the packages, jobs and revisions within your Matillion ETL instance.

    All the APIs listed in this section are available to use with GET/POST/DELETE methods to retrieve the data used to get, update, import/export, delete the resource within the shared jobs.

    List of endpoints for the /sharedjob:

    Below is the detailed description of these endpoints with example responses from the server.

    GET/export

    To export the shared jobs metadata from within your Matillion instance, use the /export endpoint after /sharedjob. This example will use the GET method REST API call to export the shared jobs metadata available within the instance.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/export
    

    Server Response:

    {
    "objects": [
    {
    "metadata": {
    "identifier": {... }
    ]
    },...
    "orchestrationJobs": [
    "Shared Jobs - Unicode"
    ],...
    "transformationJobs": [],       
    },
    "variables": {
    "oauth": {....}
    },  ....      
    ],
    "version": "master",
    "environment": "redshift"
    }
    

    POST/import

    Now you have already export the shared job metadata in the example before. This time is to import the data via ExportContainer. Supply the data as already encoded since the shared job will be created as-is. This will be a POST method API call, as we will have to attach the imported data in the body as a JSON file to run into the Matillion ETL instance.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/export
    

    Server Response:

    {
    "name": "sharedjobs",
    "statusList": [
    {
    "success": true,
    "name": "matillion.regression.Unicode_1.1"
    },
    {
    "success": true,
    "name": "Matillion.Incremental.Gmail Incremental Load.1",
    }
    ],....
    "success": true
    }
    

    GET/package

    To get the packages available within the shared job, use the /package endpoint after /sharedjob. This example will use the GET method REST API call to fetch the package details available within the shared jobs.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/export
    

    Server Response:

    [
    "TestPackage-1",
    "TestPackage-2",
    "TestPackage-3"
    ]
    

    PATH/package/name/{packageName}

    List of endpoints for the /package/name/{packageName}:

    Below is the detailed description of these endpoints with example responses from the server.

    GET/export

    To export a specific package data from within the shared jobs, provide /{packageName}, and use the /export endpoint after that. This example will use the GET method REST API call to export a package available within your Matillion ETL instance.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/export
    

    Server Response:

    {
    "objects": [
    {
    "metadata": {...
    },
    "orchestrationJobs": {...}
    },
    "transformationJobs": {...},       
    },
    ],
    "version": "master",
    "environment": "redshift"
    }
    

    GET/job

    This is an example of the GET method REST API call to retrieve the jobs available within the selected package.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job
    

    Server Response:

    [
    "SharedJob"
    ]
    

    POST/delete

    This will be a POST method API call. The /delete endpoint will allow you to delete the selected package from the shared job, indicating the revision id.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/delete
    

    Server Response:

    {
    "success": true,
    "msg": "Successfully deleted [3] Shared Job(s).",
    "id": -1
    }
    

    DELETE/{packageName}

    To remove any resource from the list, we will use the DELETE API request. In this example, we will delete a package from the shared job.

    Note

    There are often multiple ways of achieving the same task within the API as resources and methods branch out and overlap.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}
    

    Server Response:

    {
    "success": true,
    "msg": "Successfully deleted [4] Shared Job(s).",
    "id": -1
    }
    

    PATH/job/name/{jobName}

    List of endpoints for the /job/name/{jobName}:

    Below is the detailed description of these endpoints with example responses from the server.

    GET/export

    To export a specific job metadata from within the shared job package, provide /{jobName} and just use the /export endpoint after that. This example will use the GET method REST API call to export a job available within the instance.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/export
    

    Server Response:

    {
    "objects": [
    {
    "metadata": {...
    },
    "orchestrationJobs": {...}
    },
    "transformationJobs": {...},       
    },
    ],
    "version": "master",
    "environment": "redshift"
    }
    

    GET/revision

    This is an example of the GET method REST API call to get the revision id associated with the current job.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/revision
    

    Server Response:

    [
    1
    ]
    

    POST/delete

    This will be a POST method API call. The /delete endpoint will allow you to delete the selected package from the shared job.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/delete
    

    Server Response:

    {
    "success": true,
    "msg": "Successfully deleted [1] Shared Job(s).",
    "id": -1
    }
    

    DELETE/{packageName}

    To remove any resource from the list, we will use the DELETE API request. In this example we will delete a job.

    Note

    There are often multiple ways of achieving the same task within the API as resources and methods branch out and overlap.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/delete
    

    Server Response:

    {
    "success": true,
    "msg": "Successfully deleted [1] Shared Job(s).",
    "id": -1
    }
    

    PATH/revision/id/{revisionId}

    List of endpoints for the /revision/id/{revisionId}:

    Below is the detailed description of these endpoints with example responses from the server.

    GET/export

    To export the job metadata using revision id from within the shared job package, provide /{revisionId} and just use the /export endpoint after that. This example will use the GET method REST API call to export a job metadata, including all of its PATHS and data types within your Matillion ETL instance.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/revision/id/{revisionId}/export
    

    Server Response:

    {
    "endpoints": [
    {...
    }
    ]'
    "dataTypes": [...
    ]
    }
    

    POST/delete

    This will be a POST method API call. The /delete endpoint will allow you to delete the selected job using its revision id from the shared job.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/revision/id/{revisionId}/delete
    

    Server Response:

    {
    "success": true,
    "msg": "Successfully deleted [1] Shared Job(s).",
    "id": -1
    }
    

    DELETE/{revisionId}

    To remove any resource from the list, we will use the DELETE API request. In this example we will delete a job .

    Note

    There are often multiple ways of achieving the same task within the API as resources and methods branch out and overlap.

    Base URL:

    http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/revision/id/{revisionId}
    

    Server Response:

    {
    "success": true,
    "msg": "Successfully deleted [1] Shared Job(s).",
    "id": -1
    }
    

    What's Next