Fix Azure Data Manager for Energy: Setup & Config Errors
Why This Is Happening
I've worked with a lot of Azure services. Azure Data Manager for Energy , ADME for short , is one of the most powerful platforms Microsoft has released for the energy sector, but it's also one of the most misunderstood from a setup and configuration standpoint. It's not a simple "spin up a VM and go" experience. It brings together the OSDU® Data Platform, Microsoft's Azure cloud infrastructure, and SLB's domain expertise into a single managed PaaS offering. That complexity is a feature. But it means there are a lot of places things can go wrong, and Microsoft's error messages rarely tell you exactly where.
Here's what I see trip people up the most. The Azure Data Manager for Energy platform is a first-party PaaS service, Microsoft manages deployment, monitoring, scaling, security, and upgrades. That's great news for your ops team, but it also means some configuration decisions are locked behind specific service principals, Microsoft Entra ID roles, and OSDU-standard entitlement structures that don't map intuitively to standard Azure RBAC. If you've ever set up an Azure SQL instance and assumed ADME would work similarly, you're going to hit a wall fast.
The most common complaints I hear are: the instance won't provision at all, data ingestion jobs fail silently or return cryptic schema validation errors, authentication tokens expire in unexpected ways, and data partition configurations don't behave the way the team expected after the initial deployment. Enterprise teams running domain-joined environments also frequently hit network-level problems when Private Links aren't configured before the first data load attempt.
Beyond the setup phase, the Domain Data Management Services (DDMS) layer adds another dimension. DDMS is how you actually interact with specialized data types, seismic, wellbore, well delivery, Petrel, Reservoir, and each DDMS has its own set of governed schemas, entitlement requirements, and connection expectations against core OSDU services like storage, legal tags, and entitlements. Miss any one of those and you'll get access denied errors or silent failures that look like data loss but aren't.
I know this is frustrating, especially when your geoscience teams are waiting on data access and you're staring at an ARM deployment error with no obvious next move. That's exactly why this guide exists. Browse all Microsoft fix guides →
Let's fix it systematically, starting with the fastest wins and working toward the deeper infrastructure issues that require a bit more care.
The Quick Fix, Try This First
Before you spend an hour digging through Azure Monitor logs or re-reading the OSDU spec, run through this checklist. It resolves the majority of Azure Data Manager for Energy setup problems I've seen in practice.
Check your subscription registration. ADME requires several Azure resource providers to be registered before the instance creation will succeed. Open the Azure Portal, go to your subscription, click Settings > Resource providers, and search for each of these. If any show as "NotRegistered", click it and hit Register:
Microsoft.OpenEnergyPlatform
Microsoft.Storage
Microsoft.KeyVault
Microsoft.Network
Microsoft.ManagedIdentity
Microsoft.AuthorizationGive it two to three minutes after registering, the portal updates asynchronously and sometimes shows a stale state.
Verify your Microsoft Entra ID tenant is configured correctly. Azure Data Manager for Energy uses Microsoft Entra ID (formerly Azure Active Directory) for all authentication and authorization. The most common quick-fix here is confirming that the service principal being used for deployment has Contributor role at the resource group level and that your Entra ID app registration has the correct redirect URIs and API permissions. Open Azure Active Directory > App registrations, find your ADME-linked app, click API permissions, and confirm the permissions are granted, not just added, but actually granted consent by an admin.
Confirm your region supports ADME. Not every Azure region has Azure Data Manager for Energy available. If your ARM template or portal deployment is failing immediately without a useful error, check the Azure Products by Region page and confirm availability before anything else. Deploying to an unsupported region produces an unhelpful "ResourceNotFound" or "Location not valid" error that looks like a template bug.
Look at Activity Log before anything else. In the Azure Portal, navigate to your resource group, click Activity log, and filter for the past hour. Failed deployments almost always surface a more specific error here, even when the deployment blade itself just shows a generic failure message.
This is the step most people skip because they assume their Azure admin already handled it. Nine times out of ten, they haven't, at least not completely. Azure Data Manager for Energy setup problems at this stage produce errors like AuthorizationFailed or MissingSubscriptionRegistration, both of which are fixable in under five minutes.
Open the Azure Portal and navigate to Subscriptions > [Your Subscription] > Settings > Resource providers. Use the filter bar and search for OpenEnergyPlatform first. If it shows "NotRegistered", select it and click Register at the top. Then do the same for Microsoft.Storage, Microsoft.KeyVault, Microsoft.Network, and Microsoft.ManagedIdentity. You need all of them.
For permissions, the account performing the deployment needs at minimum Owner or Contributor + User Access Administrator at the subscription or resource group level. ADME creates managed identities during provisioning and assigns roles to them, that step fails silently if the deploying principal can't create role assignments.
You can confirm your own role assignment quickly with Azure CLI:
az role assignment list --assignee [your-user-or-sp-object-id] \
--scope /subscriptions/[subscription-id] \
--output tableIf you see Contributor but not User Access Administrator or Owner, talk to your Azure admin before going further. Trying to work around missing role assignment permissions will cost you hours of failed deployments.
When permissions are right and providers are registered, your next deployment attempt should move past the initial authorization phase. You'll see the ADME resource appear in the portal under your resource group, that's your signal it worked.
Creating an Azure Data Manager for Energy instance is not the same as creating most other Azure resources. There are several fields that, once set, cannot be changed, and getting them wrong means deleting and restarting. Pay attention here.
In the Azure Portal, search for "Azure Data Manager for Energy" in the top search bar and click Create. You'll be prompted for:
- Subscription, make sure this is the same subscription where you registered the resource providers
- Resource group, create a new, dedicated resource group for ADME; don't share it with other services unless you have a deliberate reason
- Name, lowercase alphanumeric, 3–24 characters, globally unique; this becomes part of your service endpoint URL
- Region, pick a supported region closest to your data consumers
- Data partition name, lowercase only, no special characters; this is the default partition created at instance setup
On the Identity tab, you'll need to link a Microsoft Entra ID application registration. If you haven't created one yet, go to Azure Active Directory > App registrations > New registration first. Set the redirect URI type to Web and add https://[your-adme-instance-name].energy.azure.com as a redirect URI. Come back to the ADME creation wizard after that's done.
Hit Review + Create. The actual deployment takes 30 to 60 minutes, this is normal. Don't cancel it early. If it fails, go immediately to the Activity Log on the resource group for the actual error details.
Once provisioning completes, navigate to the resource. Under Settings > Data Partitions, confirm your default partition shows as Active. If it's missing or shows a different status, you'll need to create it manually via the management API before ingesting any data.
Authentication is where Azure Data Manager for Energy OSDU platform access most commonly breaks down after a successful deployment. The platform uses Microsoft Entra ID for all auth, no legacy API key flows. Every API call you make against ADME or its DDMS endpoints needs a valid bearer token, and that token must come from the Entra ID app registration you linked at setup time.
To generate an auth token manually for testing, use the following Azure CLI command (you'll need the Azure CLI installed and the energy extension):
az account get-access-token \
--resource [your-client-id-from-app-registration]Copy the accessToken value from the output. This token is valid for one hour by default. Pass it in your API requests as:
Authorization: Bearer [access-token]The most common auth error is 401 Unauthorized with a body that just says "token validation failed." This almost always means one of three things: the token audience doesn't match the ADME client ID, the app registration doesn't have the correct API permissions granted, or the user or service principal making the request hasn't been added to the ADME entitlements groups yet.
For that last one, entitlements are managed separately from Azure RBAC. Even if your Azure account has Owner on the resource group, you still need to be added to the OSDU entitlement groups within ADME itself. Use the Entitlement Service API (documented under Administration > How to manage users) to add your user to the users@[partition-id].[instance-name].dataservices.energy group. Without this, you'll get 401s on every OSDU API call regardless of your Azure permissions.
Once auth is working, all API calls should return HTTP 200 or appropriate OSDU response codes. A successful auth token validation will return your user's group memberships in the entitlement service response, that confirms end-to-end auth is configured correctly.
One of the genuinely great features of Azure Data Manager for Energy is that it supports multiple data partitions per instance, and you can add more after initial deployment. But this architecture also means data access controls operate at two levels simultaneously: Azure RBAC (who can touch the ADME resource itself) and OSDU ACLs (who can read or write specific records within a partition). Mixing these up is the source of a lot of "I have permissions but still can't access data" tickets.
To add a new data partition after initial instance creation, go to your ADME resource in the portal and click Settings > Data Partitions > + Add. The name must be lowercase alphanumeric. After clicking Add, the new partition takes about 10–15 minutes to become active. Don't attempt to ingest data into it until it shows Active status.
Legal tags are mandatory for every record you store in ADME. Without a valid legal tag attached to your data, the ingestion will fail at the legal service validation step. Create legal tags before you run any ingestion jobs. Via the portal, navigate to OSDU Services or hit the Legal Service API directly:
POST https://[instance].energy.azure.com/api/legal/v1/legaltags
Content-Type: application/json
Authorization: Bearer [token]
data-partition-id: [your-partition-id]
{
"name": "[partition-id]-my-legal-tag",
"description": "Legal tag for seismic data",
"properties": {
"contractId": "A1234",
"countryOfOrigin": ["US"],
"dataType": "Public Domain Data",
"exportClassification": "EAR99",
"originator": "MyCompany",
"personalData": "No Personal Data",
"securityClassification": "Public"
}
}ACLs on individual records use the OSDU group format: viewers@[partition-id].[instance].dataservices.energy and owners@[partition-id].[instance].dataservices.energy. Use the Entitlement Service to create these groups and populate them before your data ingestion runs. If viewers or owners groups don't exist when a record is stored, the storage service will return a 400 error referencing an invalid ACL.
When everything is configured correctly, a test record write to the Storage Service will return HTTP 201 with a record ID in the response body. That's your confirmation that partitions, ACLs, and legal tags are all working together properly.
Data ingestion is where Azure Data Manager for Energy OSDU configuration errors surface in the most visible and painful way. You've set everything up, you kick off a manifest ingestion job or CSV parser run, and it fails with a schema validation error or just sits in a "Running" state indefinitely. I've been there. Here's how to diagnose it.
For Manifest Ingestion failures, the first thing to check is your manifest file structure. The manifest must conform to the OSDU manifest schema for the specific entity type you're ingesting (Master Data, Reference Data, or Work Product). Schema validation failures appear in the Workflow Service logs. Check the job status via the Workflow API:
GET https://[instance].energy.azure.com/api/workflow/v1/workflow/[workflow-name]/workflowRun/[run-id]
Authorization: Bearer [token]
data-partition-id: [partition-id]Look at the message field in the response. Common specific errors: schema not found for kind means the schema hasn't been loaded into your partition yet, you need to run the schema initialization workflow first. legal tag not found means your manifest references a legal tag that doesn't exist yet in the partition (see Step 4).
For CSV Parser failures, the error is almost always one of three things: the CSV doesn't match the expected column headers for the target schema kind, the date/time format in the CSV doesn't match ISO 8601, or the file wasn't uploaded correctly to the File Service before triggering the parser workflow. The CSV parser in ADME expects the file to already be staged in the ADME File Service, not in an arbitrary Azure Blob Storage container.
Upload your CSV to the File Service first:
POST https://[instance].energy.azure.com/api/file/v2/files/uploadURL
Authorization: Bearer [token]
data-partition-id: [partition-id]This returns a pre-signed upload URL. PUT your file to that URL, then use the returned file ID in your CSV parser workflow trigger. If you skip the File Service upload step and try to point the parser directly at a blob URI, it will fail with a permissions error every time.
Once a successful ingestion completes, you should be able to query your records through the Search Service and see them returned with the correct kind, acl, and legal attributes populated. That's full end-to-end ingestion confirmed.
Advanced Troubleshooting
Diagnosing Issues via Azure Monitor Log Integration
Out of the box, ADME doesn't expose raw service logs to you, it's a managed platform. But you can and should integrate logs from Airflow, Elasticsearch, and core OSDU services with Azure Monitor. Do this before you need it, not after a problem hits production.
In the Azure Portal, go to your ADME resource and click Monitoring > Diagnostic settings > + Add diagnostic setting. You can route logs to a Log Analytics workspace, a storage account, or an Event Hub. For real-time debugging I recommend the Log Analytics workspace route. Once connected, you can query logs in Kusto Query Language (KQL). A starting query for ingestion errors:
AzureDiagnostics
| where ResourceType == "OSDUDATAPLATFORMS"
| where Level == "Error"
| order by TimeGenerated desc
| take 100For Airflow DAG failures specifically (which power the manifest ingestion and CSV parser workflows), integrate Airflow logs per the official guidance. Common DAG failures that only show up in Airflow logs and nowhere else include: task timeouts hitting the default 30-minute limit on long ingestion jobs, schema service calls returning 404 because a schema version wasn't bootstrapped, and storage service 409 conflicts when the same record ID is written twice.
Private Link and Network-Level Issues
Enterprise deployments almost always need Private Links to prevent ADME endpoints from being exposed to the public internet. If you're seeing intermittent connection failures, or your internal applications can't reach the ADME APIs from within a VNet, Private Link is likely the fix.
Navigate to ADME Resource > Settings > Private endpoint connections > + Private endpoint. Create the private endpoint in the same VNet as your consuming applications. You also need to configure private DNS zones for the ADME endpoint, without DNS resolution pointing to the private IP, your internal apps will still route to the public endpoint.
The DNS zone name to create is privatelink.energy.azure.com. Link it to your VNet. After that, an nslookup [instance].energy.azure.com from inside the VNet should resolve to a private IP in the 10.x.x.x range rather than a public Azure IP.
Customer-Managed Encryption Keys (CMEK) Failures
If you've enabled CMEK via Azure Key Vault and ADME starts returning errors after a Key Vault operation, check two things immediately: first, confirm that the ADME managed identity still has Key Vault Crypto Service Encryption User role on the Key Vault. Key Vault access policy changes or RBAC changes can silently break CMEK. Second, make sure the key hasn't been rotated without updating the ADME key reference, ADME doesn't automatically follow key version rotations unless you've configured key auto-rotation and updated the key reference in ADME settings.
DDMS Connectivity and API Errors
Each DDMS, Seismic, Wellbore, Well Delivery, Petrel, Reservoir, Rock and Fluid Samples, has its own API surface and its own set of entitlement group requirements. A common mistake is treating DDMS access as purely an Azure RBAC matter. It isn't. You need the right OSDU entitlement groups, and some DDMS (like Seismic DDMS) have additional storage tier considerations for large seismic workloads. If you're working with SEG-Y data, review the documented conversion paths to oVDS or ZGY formats before uploading, uploading raw SEG-Y without conversion will result in data that's stored but largely inaccessible for most downstream workflows.
Prevention & Best Practices
The best Azure Data Manager for Energy troubleshooting is the kind you never have to do. I've watched teams spend weeks fixing problems that were entirely avoidable with a bit of upfront planning. Here's what separates a smooth ADME deployment from a painful one.
Bootstrap schemas and reference data before any production ingestion. ADME ships with OSDU schemas available, but they need to be loaded into your specific data partitions before the ingestion workflows can use them. Run the schema initialization workflow (documented in the OSDU services section of the official docs) for each partition during setup. Skipping this means every manifest ingestion will fail at schema validation until you fix it later under production pressure.
Set up Azure Monitor diagnostic logging on day one. Not after problems appear. Configure the Log Analytics workspace integration for Airflow, Elasticsearch, and OSDU service logs as part of your initial deployment runbook. The logs are invaluable for everything from capacity planning to debugging ingestion failures, and you can't retroactively collect logs from before the integration was enabled.
Manage entitlement groups as code. OSDU entitlement group membership is one of those things that sprawls quickly when managed manually. Put your entitlement group creation and user assignment scripts into source control and run them through your CI/CD pipeline. This makes onboarding new team members and auditing access much more manageable than clicking through the Entitlement Service API one call at a time.
Test your ingestion pipeline with a small synthetic dataset before loading production data. The ADME CSV parser and manifest ingestion workflows are strict about schema conformance. Running a five-record test file through the full pipeline, upload to File Service, trigger workflow, query via Search Service, before your first real data load will surface configuration gaps when fixing them is easy rather than urgent.
Plan your data partition strategy before deployment. You can add partitions after the fact, but you can't move data between partitions without re-ingesting it. Think about your data governance boundaries, regional requirements, and project isolation needs before you create your first partition. A well-thought-out partition structure from day one saves a lot of painful data migration work later.
- Register all required Azure resource providers before starting ADME deployment, do it as a pre-flight checklist item, every time
- Create a dedicated Entra ID app registration for ADME and document the client ID, tenant ID, and redirect URIs in your team's runbook immediately
- Set up Private Links during initial deployment, not after, retrofitting private networking onto a live ADME instance that's already serving traffic is significantly more disruptive
- Keep a named "break-glass" admin account in the OSDU entitlement owners group that your team can fall back on if a service principal misconfiguration locks everyone else out
Frequently Asked Questions
What exactly is Azure Data Manager for Energy and how is it different from regular Azure storage?
Azure Data Manager for Energy is a fully managed PaaS offering built specifically for the energy industry. It's not just a storage layer, it's an entire data platform that packages the OSDU® Data Platform standard, Microsoft Entra ID-based auth, domain-specific data management services (DDMS) for seismic, wellbore, and reservoir data, and built-in integration with SLB tools like Petrel. Regular Azure Blob Storage or Azure SQL just holds data. ADME understands energy domain data schemas, enforces legal tags and entitlements, runs ingestion workflows, and provides indexed search across your energy datasets. The two are genuinely not comparable for energy industry workloads.
How long does it actually take to create an Azure Data Manager for Energy instance?
Plan for 30 to 60 minutes for the initial instance provisioning to complete. This is not a bug or a stuck deployment, ADME is standing up a significant amount of underlying infrastructure including OSDU services, managed identities, storage accounts, and Key Vault integrations on your behalf. If your deployment has been running for more than 90 minutes without completing, check the Activity Log on your resource group for errors. A deployment that's genuinely stuck rather than just running will typically show an error event in the Activity Log within 60 minutes.
Why do I keep getting 401 Unauthorized even though I have Owner role on the Azure resource group?
Azure RBAC and OSDU entitlements are two separate permission systems that don't talk to each other automatically. Having Owner on the Azure resource group means you can manage the ADME resource itself, create it, delete it, configure it. But to actually call OSDU APIs and read or write data, your user or service principal needs to be a member of the appropriate OSDU entitlement groups inside ADME. Use the Entitlement Service API to add yourself to the users@ group for your data partition. Also verify your auth token's audience matches your ADME app registration's client ID, a token generated for a different resource will fail validation every time even if everything else is correct.
Can I add more data partitions after my Azure Data Manager for Energy instance is already running?
Yes, and this is one of the genuinely useful features of ADME. Go to your ADME resource in the Azure Portal, navigate to Settings > Data Partitions, and click + Add. Give the new partition a lowercase alphanumeric name and wait 10 to 15 minutes for it to become Active. After that, you'll need to initialize schemas and entitlement groups for the new partition separately, they don't inherit from your existing partitions automatically. One thing you can't do is merge existing partitions or move records between them without re-ingestion, so plan your partition structure carefully upfront.
My manifest ingestion job is stuck in "Running", what do I check?
First, query the Workflow Service for the specific run ID to get the detailed status message, the portal summary view doesn't show enough detail. If the message references a schema validation failure, the OSDU schemas for your entity type may not be bootstrapped in that partition yet. If it's a legal tag error, the tag named in your manifest doesn't exist in the partition. If the job has been running for more than 30 minutes with no status change at all, check your Azure Monitor Airflow logs for DAG task failures. The default task timeout in Airflow is 30 minutes, so long ingestion jobs on large datasets can silently time out without marking the workflow as failed at the API level.
What is DDMS and do I need it for every type of energy data?
Domain Data Management Services (DDMS) are specialized APIs within ADME designed for specific energy data types, seismic, wellbore, well delivery, reservoir, Petrel, and rock and fluid samples data. Each DDMS is optimized for the storage patterns, access patterns, and file formats relevant to that domain. Not all data needs a DDMS, standard reference data, master data, and work products go through the core OSDU manifest ingestion flow. But if you're working with large seismic datasets, well log streaming, or Petrel projects, the relevant DDMS is the right path because it handles bulk data performance, multi-format support (SEG-Y to ZGY and oVDS conversion for seismic, for example), and lineage tracking that the general-purpose OSDU storage service isn't designed for.