AWS Glue

Glue Python Shell job module import error

By Sai Kiran Pandrala · Last verified: 2026-05-31 · Source: AWS re:Post, community Q&A, AWS docs

At a glance

Service	AWS Glue
Cloud	Amazon Web Services (AWS)
Guide type	Procedure
Skill level	Intermediate to advanced
Time	15 - 60 minutes depending on account size

Glue Python Shell job module import error on AWS Glue sits in the most-reported issues list across r/aws, AWS re:Post, and StackOverflow. The recovery path is mostly known, the AWS docs just bury it under three layers of conceptual material.

What glue python shell job module import error actually involves on AWS Glue

Real-world context. Budget honestly for ~Rs 0 INR for the fix itself, support plan adds Rs 2,500 to Rs 1,00,000 INR per month (around $30 to $1,200 USD/month), because the cheap path looks tempting until a part shows up wrong. You will burn ~15 to 45 minutes hands-on and roughly ~1 to 4 hours including IAM review and post-fix validation once verification is done. Before you touch anything, line up an admin IAM role, the AWS CLI v2, and a CloudTrail filter pointed at the affected resource — those three are what saves you when the first attempt does not stick.

This task on AWS Glue is one of the more searched operational topics on AWS in the last 12 months. The procedure below is the path that works in a current AWS account with default IAM and standard VPC config.

The rest of this page is the structured fix path. Start with diagnose, then remediation, then the automation options so you do not have to do this by hand the next time it surfaces. Verify and safety sections at the end are the discipline that keeps the fix from regressing in production.

Signal review

Reproduce the failure with the AWS CLI in --debug mode. The full SigV4 request payload it emits, plus the exact endpoint URL it resolved to, is what AWS Support uses to verify policy, region, or parameter issues without you having to share IAM credentials. Save the debug output to a file with aws ... --debug 2> debug.log and you can search it for the failed aws.request entry.

Look at the CloudTrail event for the failed call, even if you are not enrolled in CloudTrail Lake. The basic 90-day event history works for most diagnostic purposes and lives in the console under CloudTrail > Event history. Filter by event name (the API action) and time range; the event JSON shows the exact user identity, source IP, request parameters, and error code.

Pull the AWS request ID from the response headers: x-amz-request-id for most services, x-amzn-RequestId for API Gateway, both x-amz-request-id and x-amz-id-2 for S3. AWS Support needs these IDs to look up your call in their internal logs - without them, the first reply on a ticket will ask you to reproduce the call and capture them. Save them with a timestamp; AWS Support cannot retrieve calls older than 90 days for most services.

Solution-focused remediation path

For IAM and STS issues, the timing matters. STS sessions can take up to 60 seconds to propagate after creation. The first call right after assume-role can fail with a permission error even when the policy is correct. Add a small retry with backoff before treating the first failure as definitive.

Most AWS Glue failures fall into one of three buckets: IAM permission gap, networking path break (security group, NACL, or VPC endpoint policy), or service-limit / quota hit. Run that mental triage first - it covers around 80 percent of real-world cases. If the failure does not fit any of the three, it is likely a service-side regression worth opening a re:Post or support ticket for.

When the failure happens in production but not in dev, do not just compare the IAM policy. Compare the SCP / RCP at the OU level, the permission boundary on the role, and the resource-based policy on the target. One of those is almost always different between accounts. AWS Config conformance packs make this comparison routine.

Automate this fix so you do not do it twice

Automate the fix with the AWS CLI

The CLI one-liner pattern for AWS Glue operations is roughly: aws glue describe-... --query ... to read state, aws glue modify-... --no-dry-run to apply the change, and aws glue describe-... --query ... again to verify. Wrap it in a shell script that sets a region variable at the top and exits on first error with set -euo pipefail so a partial run does not leave the account in a half-fixed state.

# Template - replace placeholders with your account specifics
export AWS_REGION=us-east-1
export AWS_PROFILE=prod
aws glue describe-... --query 'Resources[?Status==`FAILED`].[Id,Reason]' --output table
aws glue modify-... --resource-id RESOURCE_ID --no-dry-run
aws glue describe-... --resource-id RESOURCE_ID --query 'Status'

Wire the fix into EventBridge for self-healing

If the failure mode is recurring, automate the remediation instead of the diagnosis. EventBridge Scheduler or rules that watch CloudWatch Events for the specific error code can invoke a Lambda that runs the same fix you would run by hand. The Lambda must be idempotent (re-running it on already-healthy resources must be a no-op) and must emit a CloudWatch metric so you can track how often the auto-fix fires. A spike in auto-fix invocations is itself a signal worth alerting on.

# EventBridge rule pattern (JSON)
{ "source": ["aws.glue"], "detail-type": ["AWS API Call via CloudTrail"], "detail": { "errorCode": ["AccessDenied", "ThrottlingException"] }
}

Automate the fix with Python and boto3

For anything you do more than twice, write a small Python script. The boto3 pattern below uses paginators (so it does not blow up on accounts with thousands of resources), explicit region binding, and a dry-run flag that defaults to True. Keep the script under 100 lines; if it grows beyond that, you are building a tool and should put it behind a Lambda with proper logging.

import boto3, sys
DRY_RUN = '--apply' not in sys.argv
client = boto3.client('glue', region_name='us-east-1')
paginator = client.get_paginator('describe_...')
for page in paginator.paginate(): for item in page.get('Items', []): if item.get('Status') == 'FAILED': if DRY_RUN: print(f'[dry-run] would fix {item["Id"]}') else: client.modify_...(ResourceId=item['Id']) print(f'fixed {item["Id"]}')

Things that bite

The most common pitfall when fixing this on AWS Glue is treating it as a one-off rather than as a recurring class of incident. The same misconfiguration tends to happen again after a deployment, a role rotation, or a region migration unless the fix is codified. Add a CloudFormation hook, Service Control Policy condition, or AWS Config rule that prevents the same misconfig from being introduced again. Documentation alone does not survive turnover.

Another common trap: confirming the fix on a single resource and assuming the fleet is healthy. Loop your check across every account, region, and IAM principal that could exhibit the same symptom. If you cannot enumerate the affected scope without a script, you do not yet understand the scope.

Repair sequence

Reproduce the original symptom path. If it still surfaces in any account or region or IAM role, you have not fixed it.
Watch for 24 to 48 hours. AWS metrics and policy systems can mask issues with cached health for 6 to 12 hours, especially CloudFront and Route 53.
Run a smoke test under realistic load. Happy-path tests miss race conditions and IAM session-cache issues.
Capture the new state in a runbook so the next person on call does not have to rediscover this. Push it to Confluence or your team wiki, not into Slack.
If the fix involved a permission change, run IAM Access Analyzer one more time to confirm you did not open a separate hole while closing this one.

Safety, rollback, blast radius

Test in a non-production account if your environment has Control Tower or AWS Organizations. The cost of one sandbox account is cheaper than one rollback meeting.
Export the existing config before changing it. Most AWS Glue resources support describe + export to JSON via CLI - capture that to source control before you start.
Know your rollback path. Some AWS Glue operations are one-way (region migration, account-level feature opt-in, KMS key deletion past pending window). Confirm reversibility on the AWS doc before you commit.
Be aware of cross-service impact. IAM role changes ripple to every service trusting that role. KMS key changes break every workload depending on that key. VPC endpoint changes affect every VPC consumer of that endpoint.
Maintenance window discipline: if the change touches DNS, certificate rotation, or anything that emits TLS handshakes, line up a window with stakeholder notification, not a heroic mid-day swap.

FAQ

How long does glue python shell job module import error typically take on AWS?

For most AWS Glue environments, 15 to 60 minutes including verification. Large multi-account setups, anything touching SCPs at the Organizations level, or cross-region replication can stretch to half a day because AWS has to wait for replication and IAM session caches.

Is there a rollback path?

Yes for most AWS Glue changes. Export the existing config to JSON via aws glue describe-... first, then commit it before you change anything. A few operations are one-way (KMS key deletion past the pending window, region migration, account closure). Check the AWS doc for the specific API before you commit.

Will this affect dependent AWS services?

Often yes. AWS Glue resources are usually referenced by other workloads (Lambda, ECS tasks, IAM-bound apps, CloudFront origins, downstream pipelines). Use IAM Access Analyzer + CloudTrail to enumerate consumers before changing a shared resource.

What if my AWS Console layout does not match these steps?

AWS Console UI moves quarterly. The Console layout in this page is current as of 2026-05-31 but the underlying CLI / SDK calls do not change as fast. If the Console version differs, fall back to aws CLI or SDK calls - those almost always still work.

Where do I get AWS Support help if I am still stuck?

Open a case via the AWS Support Center with: the request ID + correlation ID, the exact error string, CloudTrail event, and your reproduction steps. AWS re:Post is the no-cost public alternative - search there first; 80% of common AWS Glue issues already have an answer with an AWS-staff-verified flag.

References

docs.aws.amazon.com - official documentation for AWS Glue
AWS re:Post (formerly forums) - community Q&A with AWS-staff-verified answers
AWS Health Dashboard at health.aws.amazon.com
AWS Service Quotas console and AWS Well-Architected Tool

Related guides worth a look while you sort this one out: