Amazon Marketing Stream (AMS) is a powerful real-time data streaming service that enables advertisers to receive near real-time advertising data directly into their systems. This guide provides a comprehensive overview of how to implement AMS for your advertising operations.
What is Amazon Marketing Stream?
Amazon Marketing Stream delivers advertising data in near real-time through the AWS Simple Queue Service (SQS), allowing businesses to:
– Monitor campaign performance instantly
– Improved data granularity
– Near real-time messaging
– More reporting dimensions
Implementation Overview
The implementation process involves several key steps that work together to establish a reliable data pipeline from Amazon’s advertising platform to your systems.
1. Infrastructure Setup with AWS CloudFormation
The foundation of AMS implementation begins with setting up the necessary AWS infrastructure:
Key Components:
– AWS SQS Queue: Acts as the message delivery endpoint
– CloudFormation Stack: Automates infrastructure provisioning
– Access Policies: Ensures secure data delivery
Process:
1. Access AWS Console and navigate to CloudFormation
2. Select the appropriate AWS region (Ireland for EU endpoints)
3. Upload the Amazon provided CloudFormation template
4. Configure stack parameters including dataset ID and queue name
5. Deploy the stack and verify SQS queue creation
2. API Subscription Management
Once infrastructure is ready, establish subscriptions through Amazon’s Advertising API:
Subscription Creation:
– Use the `/streams/subscriptions` endpoint
– Provide authentication headers (Client ID, Profile ID, Access Token)
– Specify dataset type (e.g., “sp-conversion” for Sponsored Products conversion data)
– Include the SQS queue ARN as the destination
Key Parameters:
– Client Request Token: Unique identifier for tracking
– Dataset ID: Specifies the type of advertising data
– Destination ARN: SQS queue where data will be delivered
– Notes: Description for subscription management
3. Subscription Confirmation Process
After creating a subscription, a confirmation process ensures secure data delivery:
Confirmation Steps:
1. Monitor SQS queue for confirmation messages
2. Extract the subscription confirmation URL from the message
3. Make a GET request to the confirmation URL
4. Verify subscription status changes to “ACTIVE”
This process typically takes 5-10 minutes to complete.
4. Data Processing and Integration
With active subscriptions, implement data processing mechanisms:
Data Handling:
– Set up automated message polling from SQS
– Parse incoming advertising data
– Transform data for your business requirements
– Integrate with existing analytics and reporting systems
Available Data Types
Amazon Marketing Stream supports various advertising datasets like DSP, SP, SD, SB etc.
Benefits of Implementation
Real-Time Insights
– Immediate access to campaign performance data
– Faster decision-making capabilities
– Reduced data latency compared to batch reporting
Operational Efficiency
– Automated data collection and processing
– Reduced manual reporting overhead
– Streamlined campaign optimization workflows
Competitive Advantage
– Faster response to market changes
– Enhanced campaign performance through real-time optimization
– Better customer experience through data-driven decisions
Conclusion
For detailed technical specifications and API documentation, refer to the official Amazon Advertising API documentation and the provided CloudFormation templates.
Amazon Marketing Stream provides a robust foundation for real-time advertising data integration. However, implementing this at enterprise scale introduces non-trivial engineering challenges. Amazon sellers often operate hundreds or thousands of concurrent campaigns across multiple marketplaces and product categories, generating substantial data volumes that can quickly overwhelm standard processing systems.
When working with enterprise-level Amazon sellers, the SQS queues can receive thousands of messages per hour, containing detailed performance data for extensive campaign portfolios. This volume presents unique challenges when using traditional web application frameworks like Laravel paired with MySQL databases.
We will discuss detailed solutions and strategies to overcome these challenges in the next part of this blog.