Lineages are built in the OvalEdge application, where we can look into the data movement and dataflow from source to destination (one or more).
This service deals with the following related APIs:
- Delete Lineage
- Insert Lineage
- Insert Lineage with names
- Get Lineage Column Graph
- Get Lineage Column Graph with names
- Get Lineage Graph
- Get Lineage Graph with names
- Trigger Lineage creation for Dashboard
1. Delete Lineage
Delete Lineage method deletes the Lineage and Lineage Columns with input Lineage Id.
Mandatory Field: lineageId.
| Method | URL | Parameter |
| deleteLineage | /api/deleteLineage/{lineageId} | {lineageId} |
Response
{
"status": true,
"statusCode": "lineage with id 1002 is deleted",
"statusMsg": "",
"response": {}
}
2. Delete Lineage By Object Type
This API is used to delete the lineage for the given object id and object type.
Mandatory Field: objectId, objectType.
| Method | URL | Parameter |
| deleteLineage | /api/deleteLineage/{lineageId} | {lineageId} |
Sample request
ObjectId - 45
ObjecType - oeschema/oedomain/filefolder
Sample response
{
"status": true,
"statusCode": "Lineage Deleted successfully for [objectId] = 45 and object type - oeschema",
"statusMsg": "",
"response": {}
}
3. Delete Lineage using Object Id
This Api is to delete the lineage using objectId.
Mandatory Field: objectId, objectType.
|
Method Name |
URL |
Parameters |
|
deleteLineageusingobjectId |
/api/deleteLineageusingobjectId |
ObjectId,ObjectType |
Sample Request Body
{
"ObjectId": 0
“ObjectType”: “Chart”
}
Sample Response Body
{
"status": true,
"statusCode": "",
"statusMsg": "",
"response": { }
}
4. Insert Lineage
Insert Lineage method creates a new Lineage between source and destination with the given input through the Request body, Source and Destination can be Table, File and Report.
Mandatory Field: sourceColumnId, targetColumnId, sourceId, sourceType, targetId, targetType if(refId is provided then these are mandatory fileds : refConnectionId,refName,refSchemaId).
| Method | URL |
| insertLineage | /api/insertLineage |
Request Body
{
"columns": [
{
"columnInfo": "string",
"sourceColumnId": 0, *
"sourceColumnType": "string", *
"targetColumnId": 0, *
"targetColumnType": "string"*
}
],
"lineageWiki": "string",*
"refType": "string",*
"refValue": "string",*
"sourceId": 0, *
"sourceType": "string",*
"targetId": 0, *
"targetType": "string" *
}
Response
{
"status": true,
"statusCode": "",
"statusMsg": "",
"response": {
"lineage": {
"sourceId": 1133,
"sourceType": "OETABLE",
"targetId": 1004,
"targetType": "OETABLE",
"refType": "SQL",
"refValue": "select * from lineage",
"columns": [
{
"sourceColumnId": 2407,
"sourceColumnType": "OETABLE",
"targetColumnId": 2662,
"targetColumnType": "OETABLE",
"columnInfo": "Info SourceRef Column Test"
}
],
"lineageWiki": "Lineage wiki 1133-1004 lineage"
}
}
}
5. Insert Lineage With Names
This API builds lineage with source and target with names as input.
Mandatory Fields
- sourceConnectionName
- sourceSchema
- sourceName
- sourceType
- targetConnectionName
- targetSchema
- targetName
- targetType
- refType
- refValue
- sourceColumnName
- targetColumnName
- refConnectionName
- refName
- refSchema
- sourceNamemsourceSchema
| Method | URL |
| insertLineageWithNames | /api/insertLineageWithNames |
Request Body
{
"status": true,
"statusCode": "",
"statusMsg": "",
"response": {
"lineage": {
"sourceId": 1133,
"sourceType": "OETABLE",
"targetId": 1004,
"targetType": "OETABLE",
"refType": "SQL",
"refValue": "select * from lineage",
"columns": [
{
"sourceColumnId": 2407,
"sourceColumnType": "OETABLE",
"targetColumnId": 2662,
"targetColumnType": "OETABLE",
"columnInfo": "Info SourceRef Column Test"
}
],
"lineageWiki": "Lineage wiki 1133-1004 lineage"
}
}
}
6. Get Lineage Column Graph
Mandatory Field:objectId, objectType.
| Method | URL | Parameter |
| getLineageColumnGraph | /api/lineagecolumngraph | {objectId},{objectType} |
Response
{
"status": true,
"statusCode": "lineage is created with lineage id 13", "statusMsg": "",
"response": {
"lineage": {
"sourceId": 1000,
"sourceType": "OETABLE",
"targetId": 1906,
"targetType": "OEFILE",
"refType": "SQL",
"refValue": "select * from datedim",
"columns": [
{
"sourceColumnId": 1004,
"sourceColumnType": "OETABLE",
"targetColumnId": 1,
"targetColumnType": "OEFILE",
"columnInfo": ""
}
],
"lineageWiki": "string"
}
}
}
7. Get Lineage column graph with names
This API is used to get the lineage graphs of both upstream (parent) and downstream (child) objects of given column details.
Mandatory Field: connectionName, objectName, objectType, schemaName, tableName.
|
Method |
Url |
Parameter |
|
lineagegraphwithnames |
/api/lineagecolumngraphwit hnames?connectionName= {connectionName}&objectN ame={objectName}&object Type={objectType}&schem aName={schemaName}&ta bleName={tableName} |
connectionName={connectionN ame}&objectName={objectNam e}&objectType={objectType}&sc hemaName={schemaName}&ta bleName={tableName} |
Response
{
"status": true,
"statusCode": "",
connectionName={connectionName}&objectName={objectName}&objectType={objectType}&schemaName={schemaName}&tableName={tableName}
"statusMsg": "",
"response": {
"lineageGraph": {
"queryType": null,
"dbName": "dbo",
"name": "dailysales",
"tableid": 1006,
"fileid": 0,
"fileLocation": null,
"node": "root",
"downstreamObjects": [
{
"queryType": null,
"dbName": "MSSQL.SuperStoreDWH.dbo", "name": "dailysales",
"tableid": 1017,
"fileid": 0,
"fileLocation": null,
"node": "child",
"downstreamObjects": null,
"upstreamObjects": null,
"isParent": false,
"nodeValue": false,
"datasetid": 0,
"datasetname": null,
"externalType": "OETABLE",
"objectType": "TABLE",
"objectid": 0,
"objectName": null,
"objectProp": null,
"lineageId": 1,
"lineageExtId": 1,
"columnId": 0,
"inputArray": null,
"inputLineageIds": null,
"level": 0,
"connectionName": "Dremio",
"columnName": null,
"parentId": "1006TABLE",
"childId": null,
"maxNode": 2,
"steward": null,
"chartid": 0,
"hasMoreLineageLevel": false,
"inProject": true
},
{
"queryType": null,
"dbName": "/",
"name": "affiliated_with.csv",
"tableid": 0,
"fileid": 1906,
"fileLocation": "/hospital-demo-bucket/", "node": "child",
"downstreamObjects": null,
"upstreamObjects": null,
"isParent": false,
"nodeValue": false,
"datasetid": 0,
"datasetname": null,
"externalType": "OEFILE", "objectType": "FILE",
"objectid": 0,
"objectName": null,
"objectProp": null,
"lineageId": 3,
"lineageExtId": 3,
"columnId": 0,
"inputArray": null,
"inputLineageIds": null,
"level": 0,
"connectionName": "S3", "columnName": null,
"parentId":"1006TABLE", "childId": null,
"maxNode": 2,
"steward": null,
"chartid": 0,
"hasMoreLineageLevel": false, "inProject": false
}
],
"upstreamObjects": null,
"isParent": null,
"nodeValue": false,
"datasetid": 0,
"datasetname": null,
"externalType": "OETABLE", "objectType": "TABLE",
"objectid": 0,
"objectName": null,
"objectProp": null,
"lineageId": 0,
"lineageExtId": 0,
"columnId": 0,
"inputArray": null,
"inputLineageIds": null,
"level": 0,
"connectionName": "Sql Server", "columnName": null,
"parentId": null,
"childId": null,
"maxNode": 2,
"steward": null,
"chartid": 0,
"hasMoreLineageLevel": false, "inProject": true
}
}
}
8. Get Lineage Graph
Get Lineage Graph method returns Lineage Graph with Object Id and Object type.
Mandatory Field: objectId, objectType.
| Method | URL | Parameter |
| getLineageGraph | /api/lineagegraph | {objectId},{objectType} |
Response
{
"status": true,
"statusCode": "",
"statusMsg": "",
"response": {
"lineageGraph": {
"queryType": null,
"dbName": "ovaledgedb",
"name": "images",
"tableid": 1117,
"fileid": 0,
"node": "root",
"children": [
{
"queryType": null,
"dbName": null,
"name": "SQL#1021",
"tableid": 0,
"fileid": 0,
"node": "child",
"children": [
{
"queryType": null,
"dbName": "ovaledgedb",
"name": "a_dqruleobject",
"tableid": 1002,
"fileid": 0,
"node": "child",
"children": null,
"parents": null,
"isParent": false,
"nodeValue": false,
"datasetid": 1021,
"datasetname": null,
"externalType": "OETABLE",
"objectType": "TABLE",
"objectid": 0,
"objectName": null,
"objectProp": null,
"lineageId": 1000,
"lineageExtId": 1000,
"columnId": 0,
"inputArray": null,
"inputLineageIds": null,
"level": 0,
"connectionName": "mysql",
"columnName": null,
"parentId": "1021DATASET",
"childId": null,
"maxNode": 1,
"steward": null,
"chartid": 0,
"inProject": false
}
],
"parents": null,
"isParent": false,
"nodeValue": true,
"datasetid": 1021,
"datasetname": null,
"externalType": "DATASET",
"objectType": "DATASET",
"objectid": 0,
"objectName": null,
"objectProp": null,
"lineageId": 1000,
"lineageExtId": 1000,
"columnId": 0,
"inputArray": null,
"inputLineageIds": null,
"level": 0,
"connectionName": null,
"columnName": null,
"parentId": "1117TABLE",
"childId": null,
"maxNode": 1,
"steward": null,
"chartid": 0,
"inProject": false
}
],
"parents": null,
"isParent": null,
"nodeValue": false,
"datasetid": 0,
"datasetname": null,
"externalType": "OETABLE",
"objectType": "TABLE",
"objectid": 0,
"objectName": null,
"objectProp": null,
"lineageId": 0,
"lineageExtId": 0,
"columnId": 0,
"inputArray": null,
"inputLineageIds": null,
"level": 0,
"connectionName": "mysql",
"columnName": null,
"parentId": null,
"childId": null,
"maxNode": 1,
"steward": null,
"chartid": 0,
"inProject": false
}
}
}
9. Get lineage graph with names
This API is used to get the lineage graphs of both upstream (parent) and downstream (child) objects of a given input object.
Mandatory Field: connectionName, objectName, objectType, schemaName.
|
Method |
Url |
Parameter |
|
lineagegraphwithnames |
/api/lineagegraphwithname s?connectionName={conne ctionName}&objectName={ objectName}&objectType={ objectType}&schemaName ={schemaName} |
connectionName={connectionN ame}&objectName={objectNam e}&objectType={objectType}&sc hemaName={schemaName} |
Response
{
"status": true,
"statusCode": "",
"statusMsg": "",
"response": {
connectionName={connectionName}&objectName={objectNam e}&objectType={objectType}&sc hemaName={schemaName}
"lineageGraph": {
"queryType": null,
"dbName": "dbo",
"name": "dailysales",
"tableid": 1006,
"fileid": 0,
"fileLocation": null,
"node": "root",
"downstreamObjects": [
{
"queryType": null,
"dbName": "MSSQL.SuperStoreDWH.dbo", "name": "dailysales",
"tableid": 1017,
"fileid": 0,
"fileLocation": null,
"node": "child",
"downstreamObjects": null,
"upstreamObjects": null,
"isParent": false,
"nodeValue": false,
"datasetid": 0,
"datasetname": null,
"externalType": "OETABLE",
"objectType": "TABLE",
"objectid": 0,
"objectName": null,
"objectProp": null,
"lineageId": 1,
"lineageExtId": 1,
"columnId": 0,
"inputArray": null,
"inputLineageIds": null,
"level": 0,
"connectionName": "Dremio",
"columnName": null,
"parentId": "1006TABLE",
"childId": null,
"maxNode": 2,
"steward": null,
"chartid": 0,
"hasMoreLineageLevel": false,
"inProject": true
},
{
"queryType": null,
"dbName": "/",
"name": "affiliated_with.csv",
"tableid": 0,
"fileid": 1906,
"fileLocation": "/hospital-demo-bucket/", "node": "child",
"downstreamObjects": null,
"upstreamObjects": null,
"isParent": false,
"nodeValue": false,
"datasetid": 0,
"datasetname": null,
"externalType": "OEFILE",
"objectType": "FILE",
"objectid": 0,
"objectName": null,
"objectProp": null,
"lineageId": 3,
"lineageExtId": 3,
"columnId": 0,
"inputArray": null,
"inputLineageIds": null,
"level": 0,
"connectionName": "S3",
"columnName": null,
"parentId": "1006TABLE", "childId": null,
"maxNode": 2,
"steward": null,
"chartid": 0,
"hasMoreLineageLevel": false, "inProject": false
}
],
"upstreamObjects": null,
"isParent": null,
"nodeValue": false,
"datasetid": 0,
"datasetname": null,
"externalType": "OETABLE", "objectType": "TABLE",
"objectid": 0,
"objectName": null,
"objectProp": null,
"lineageId": 0,
"lineageExtId": 0,
"columnId": 0,
"inputArray": null,
"inputLineageIds": null,
"level": 0,
"connectionName": "Sql Server",
"columnName": null,
"parentId": null,
"childId": null,
"maxNode": 2,
"steward": null,
"chartid": 0,
"hasMoreLineageLevel": false,
"inProject": true
}
}
}
10. Trigger Lineage Creation For Dashboard
This API triggers lineage creation for the given dashboard or for the dashboards from the given connection.
Mandatory Field: chartId,chartName, connectionId, clientName, connectionName, domainId, domainName, domainNames.
| Method | URL |
| triggerLineageCreationForDashboard | /api/triggerLineageCreationForDashboard |
Request Body
{
"connectionId": 123,
"connectionName": "string",
"domainId": 456,
"domainName": “string”,
“chartId”: 123,
“chartName”: “string”,
“clientName”: “string”
}
Response Body
{
"status": “”,
"statusCode": "",
"statusMsg": “”,
"response": {
“Job Schedule Info”: “Job is scheduled with job id : 1234”
}
}