Context-Driven Test Data Generation using Bedrock API in Test Automation
Problem: Manual test data creation is time-consuming, error-prone, and lacks real-world variability. Static datasets limit coverage, and crafting edge cases demands significant engineering effort. Automation often fails due to outdated or irrelevant test data that doesn't reflect evolving requirements.
Challenges: Absence of context-aware test data generation during runtime. Limited coverage of negative or edge case scenarios. Pipeline delays due to manual test data prep. Need for scalable, domain-specific test data across services
Solution: Upload historical data, business rules, and domain documents to S3 . Use Amazon Bedrock APIs to process this information and generate contextual, dynamic test data with LLMs like Claude, Titan, or AI21. Integrate this directly into frameworks like Playwright, Selenium, or REST-Assured - triggered at runtime or as pre-run setup. Changes in S3 documents automatically reflect in subsequent test runs
How It Works:
Upload to S3 â Central repository for relevant docs
Chunk Data â Improve LLM efficiency
Generate Embeddings â Via Bedrock (Titan)
Store in Vector DB â Fast retrieval via similarity
Generate Data â Prompt LLMs for scenario-specific outputs
Use in Tests â Inject into automation for realistic runs
What We Tried:
Excel/SQL, Faker , fine-tuned prompts not yet integrated into AWS CodePipeline
Sidhartha Shukla
Sidharth is a Senior Software Development Engineer in Test (SDET) at Amazon with 14+ years of experience in automation engineering, developer productivity, and test strategy. He leads initiatives to integrate Generative AI across the software lifecycle-from requirements to release-driving measurable gains in engineering efficiency. A passionate mentor, Sidharth has trained over 5000 QA professionals globally and is the creator of one of the highest-selling courses on Generative AI in software testing. He frequently speaks at international conferences, with recent appearances in the USA, Canada, and India. His community contributions earned him a spotlight on the Times Square billboard. Through his writing, talks, and mentorship, Sidharth continues to shape the future of intelligent, scalable, and high-impact test automation.
Prasad Yacham
Working as Senior SDET at Amazon HQ.