Type Alias DataTaskInstanceState

Represents the execution state of a task instance, including progress metrics, errors, and operation-specific statistics.

type DataTaskInstanceState = {
    cdcStatus?: {
        accumulatingChangesCount?: number;
        applyingChangesCount?: number;
        incomingChangesCount?: number;
        latency?: string;
        throughputInKilobytesPerSecond?: number;
        totalProcessedCount?: number;
    };
    duration?: string;
    endTime?: string;
    errors?: Error[];
    fullLoad?: {
        completedCount?: number;
        errorCount?: number;
        loadingCount?: number;
        queuedCount?: number;
    };
    general?: {
        datasetCount?: number;
        datasetsInErrorCount?: number;
        dataTaskUpdatedTo?: string;
        gatewayId?: string;
        gatewayName?: string;
        gatewayTaskName?: string;
        lakehouseClusterId?: string;
        lakehouseClusterName?: string;
        liveViewsUpdatedTo?: string;
    };
    lastBatchOfChanges?: {
        relatesToRecordsFrom?: string;
        relatesToRecordsTo?: string;
        throughputInRecordsPerSecond?: number;
        totalProcessedCount?: number;
    };
    message?: string;
    startTime?: string;
    state?: | "STARTING"
    | "RUNNING"
    | "COMPLETED"
    | "FAILED"
    | "CANCELED"
    | "STOPPING";
    streaming?: {
        errorCount?: number;
        latency?: string;
        queuedCount?: number;
        runningCount?: number;
        totalProcessedCount?: number;
    };
    traceId?: string;
}
Index
cdcStatus? duration? endTime? errors? fullLoad? general? lastBatchOfChanges? message? startTime? state? streaming? traceId?
cdcStatus?: {
    accumulatingChangesCount?: number;
    applyingChangesCount?: number;
    incomingChangesCount?: number;
    latency?: string;
    throughputInKilobytesPerSecond?: number;
    totalProcessedCount?: number;
}

Change Data Capture status information for tasks performing incremental updates, including latency and processing counts.

Type Declaration

  • OptionalaccumulatingChangesCount?: number
  • OptionalapplyingChangesCount?: number
  • OptionalincomingChangesCount?: number

    Number of incoming changes. Only relevant for 'Iceberg Storage' and 'Streaming Transform' tasks.

  • Optionallatency?: string

    Duration in HH:MM:SS format (hours:minutes:seconds)

  • OptionalthroughputInKilobytesPerSecond?: number

    Throughput in kilobytes per second

  • OptionaltotalProcessedCount?: number
duration?: string

Duration in HH:MM:SS format (hours:minutes:seconds)

endTime?: string

Timestamp indicating when the task instance ended

errors?: Error[]

List of errors encountered during the last run

fullLoad?: {
    completedCount?: number;
    errorCount?: number;
    loadingCount?: number;
    queuedCount?: number;
}

Statistics for the full load phase of the task run, tracking dataset processing progress.

Type Declaration

  • OptionalcompletedCount?: number

    Number of datasets that have completed full load in this task run

  • OptionalerrorCount?: number

    Number of datasets that have failed full load in this task run

  • OptionalloadingCount?: number

    Number of datasets that are currently being loaded in this task run

  • OptionalqueuedCount?: number

    Number of datasets that are queued for full load in this task run

general?: {
    datasetCount?: number;
    datasetsInErrorCount?: number;
    dataTaskUpdatedTo?: string;
    gatewayId?: string;
    gatewayName?: string;
    gatewayTaskName?: string;
    lakehouseClusterId?: string;
    lakehouseClusterName?: string;
    liveViewsUpdatedTo?: string;
}

Overall task execution statistics including dataset counts, data freshness timestamps, and infrastructure details.

Type Declaration

  • OptionaldatasetCount?: number

    Total number of datasets produced by the task, including ones in error

  • OptionaldatasetsInErrorCount?: number

    Count of datasets that encountered errors

  • OptionaldataTaskUpdatedTo?: string

    The latest point in time the data reflects, based on updates from the source system.

  • OptionalgatewayId?: string

    For tasks that run on a gateway, this is the id of the gateway

  • OptionalgatewayName?: string

    For tasks that run on a gateway, this is the name of the gateway

  • OptionalgatewayTaskName?: string

    For tasks that run on a gateway, this is the internal name of the task on the gateway

  • OptionallakehouseClusterId?: string

    For lakehouse storage tasks, this is the id of the cluster where the task runs

  • OptionallakehouseClusterName?: string

    For lakehouse storage tasks, this is the name of the cluster where the task runs

  • OptionalliveViewsUpdatedTo?: string

    The latest point in time the live views reflect, based on updates from the source system.

lastBatchOfChanges?: {
    relatesToRecordsFrom?: string;
    relatesToRecordsTo?: string;
    throughputInRecordsPerSecond?: number;
    totalProcessedCount?: number;
}

Statistics for the most recent batch of changes processed during the task run, including timing and throughput metrics.

Type Declaration

  • OptionalrelatesToRecordsFrom?: string

    This batch starts with operational source changes from this time.

  • OptionalrelatesToRecordsTo?: string

    This batch ends with operational source changes from this time.

  • OptionalthroughputInRecordsPerSecond?: number

    Throughput in records per second

  • OptionaltotalProcessedCount?: number
message?: string
startTime?: string

Timestamp indicating when the task instance started

state?:
    | "STARTING"
    | "RUNNING"
    | "COMPLETED"
    | "FAILED"
    | "CANCELED"
    | "STOPPING"
streaming?: {
    errorCount?: number;
    latency?: string;
    queuedCount?: number;
    runningCount?: number;
    totalProcessedCount?: number;
}

Real-time streaming statistics for tasks that continuously process data streams.

Type Declaration

  • OptionalerrorCount?: number

    Number of streaming datasets that have encountered errors

  • Optionallatency?: string

    Duration in HH:MM:SS format (hours:minutes:seconds)

  • OptionalqueuedCount?: number

    Number of streaming datasets that are queued

  • OptionalrunningCount?: number

    Number of streaming datasets that are currently running

  • OptionaltotalProcessedCount?: number

    Total number of records processed

traceId?: string

Trace identifier for the last run, useful for diagnostics and support

Settings
Member Visibility

On This Page
Properties