Querying the Microsoft Security Data Lake via REST

The API endpoints, authentication scope, and request format for running KQL against Sentinel's long-term security store.

The Microsoft Sentinel data lake is the long-term retention layer behind Microsoft Sentinel and the Unified Security Operations Platform. It holds security data well beyond the standard Log Analytics retention window — up to twelve years — and exposes it through a KQL query interface. Microsoft provides access to this data through the Defender portal, but the underlying REST API is badly documented. What follows is a description of the API surface, worked out through trial and error.

The API base URL

The Data Lake API lives at api.securityplatform.microsoft.com, with regional variants for data residency. Each region has its own subdomain:

The region should match where your Sentinel workspace data resides. If in doubt, the global endpoint works but may route queries cross-region.

Authentication

This is where the Data Lake diverges from other Microsoft security APIs. The OAuth2 scope is not a URL — it is a bare application ID:

73c2949e-da2d-457a-9607-fcc665198967/.default offline_access

That GUID is a Microsoft resource identifier that the Data Lake API accepts as a token audience. It must be requested as the scope during the OAuth2 flow. The offline_access scope is needed to obtain a refresh token.

The client application ID matters too. The standard Azure CLI client (04b07795-8ddb-461a-bbee-02f9e1bf7b46) does not work here. The Data Lake requires Microsoft's Sentinel extension client:

Client ID: 4d6257d2-8e40-4f75-8da7-9cc73eab0764

With those two values, authentication follows the standard Microsoft OAuth2 v2.0 device code flow against https://login.microsoftonline.com/{tenant-id}/oauth2/v2.0/devicecode. The tenant must be a real tenant GUID, not common or organizations.

The app registration in your Entra ID tenant needs the Microsoft Threat Protection → AdvancedHunting.Read delegated permission with admin consent granted.

Three API surfaces

The Data Lake exposes three distinct endpoint groups under the base URL. All requests require an Authorization: Bearer {token} header and Accept: application/json.

Schema browsing

Two REST endpoints return metadata about available databases and tables.

List databases:

GET /lake/databases?api-version=2024-07-01

Returns a JSON object with a value array. Each entry contains databaseName, supportedDatabaseTier (either "Lake" or "Analytics"), and an optional sentinelWorkspace object with the linked workspace name and ID. Analytics-tier databases and those linked to a Sentinel workspace should be queried through a Log Analytics connection instead — the Data Lake endpoint is for lake-tier data.

List tables:

GET /lake/tables?api-version=2024-05-01-preview&databaseName={db}

Returns a paginated response with an items array of table objects (each containing name, databaseName, tableType, and a schema array of column definitions). Pagination is handled via a nextLink field — follow it until it is absent.

KQL query execution

The query endpoint uses the ADX v1 REST wire protocol, nested under the /lake/kql path:

POST /lake/kql/v1/rest/query Content-Type: application/json; charset=utf-8

The request body requires three fields — db, csl, and properties — all of which must be present:

{
  "db":  "your-database-name",
  "csl": "SecurityAlert | take 10",
  "properties": "{\"Options\":{\"servertimeout\":\"00:08:00\",\"queryconsistency\":\"weakconsistency\"}}"
}

The properties field is a JSON-encoded string, not a nested object. Standard ADX accepts both formats, but the Data Lake endpoint requires the serialised string form. Omitting the field entirely or passing an empty object causes the request to fail.

Two additional headers are expected on query requests:

x-ms-app: YourAppName x-ms-client-request-id: YourAppName;unique-request-id

The x-ms-app header identifies the calling application. The x-ms-client-request-id is used for tracing and should be unique per request. Both appear in the query statistics returned in the response.

The response follows the standard ADX v1 format — a Tables array where the first table is the primary result, followed by query properties and status tables:

{
  "Tables": [
    {
      "TableName": "PrimaryResult",
      "Columns": [
        {"ColumnName": "TimeGenerated", "DataType": "DateTime", "ColumnType": "datetime"},
        {"ColumnName": "AlertName", "DataType": "String", "ColumnType": "string"}
      ],
      "Rows": [
        ["2026-02-19T10:30:00Z", "Suspicious sign-in activity"],
        ...
      ]
    }
  ]
}

Set your HTTP client timeout generously. Data Lake queries can take significantly longer than standard ADX or Log Analytics queries — five minutes is a reasonable ceiling.

Management commands

A limited set of Kusto management commands are available at:

POST /lake/kql/v1/rest/mgmt Content-Type: application/json; charset=utf-8

The request body takes db and csl (no properties field required). Only .show version, .show databases, and .show database are supported. Other management commands return an error.

Putting it together

A minimal working example using curl, assuming you have already obtained a bearer token through the device code flow:

# List databases
curl -s \
  -H "Authorization: Bearer $TOKEN" \
  -H "Accept: application/json" \
  "https://api.securityplatform.microsoft.com/lake/databases?api-version=2024-07-01"

# Run a KQL query
curl -s -X POST \
  -H "Content-Type: application/json; charset=utf-8" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -H "x-ms-app: my-tool" \
  -H "x-ms-client-request-id: my-tool;$(uuidgen)" \
  -d '{
    "db": "your-database",
    "csl": "SecurityAlert | take 5",
    "properties": "{\"Options\":{\"servertimeout\":\"00:08:00\",\"queryconsistency\":\"weakconsistency\"}}"
  }' \
  "https://api.securityplatform.microsoft.com/lake/kql/v1/rest/query"

What is not documented here

The Data Lake also exposes a /lake/kql/jobs endpoint for asynchronous long-running queries. We have not explored this in production and cannot describe its request format with confidence.

The regional endpoint list above was derived from the Microsoft Sentinel extension client and may not be exhaustive. New Azure regions may have endpoints that are not yet reflected here.

The API versions (2024-07-01 for databases, 2024-05-01-preview for tables) are those that worked at the time of writing. Microsoft may introduce newer versions or graduate the preview endpoints.

Context

This API surface was discovered while building Data Lake connectivity for the upcoming release of Kustainer UI, an open-source security posture analysis tool.  Kustainer UI connects to ADX clusters, Sentinel workspaces, Fabric EventHouses, Azure Resource Graph, Defender XDR, and the Sentinel data lake through a unified KQL interface with device code authentication and encrypted token storage. 

I am in the process of creating final documentation of this project so keep an eye open for it being switch to public!