👋 Welcome to Order Management + Webhooks
A robust, scalable PHP application designed to handle high-volume order processing with webhook integration. This system demonstrates enterprise-level patterns for handling webhooks, ensuring idempotency, managing concurrency, and maintaining comprehensive audit trails.
📹 Demo & Resources
🎥 Code Review Video
Code Review & Technical Implementation
Duration: ~13 minutes
📖 Project Overview
This PHP application provides a comprehensive order management system with webhook processing capabilities. It's designed to handle high-pressure scenarios with 20M+ records while maintaining data integrity and performance.
Core Features
- • Order Management with Status Tracking
- • Webhook Processing with Retry Logic
- • Comprehensive Audit Trail
- • Queue Processing & Background Jobs
- • Idempotency Protection
- • Concurrency Control
Technical Highlights
- • Database Transactions with Optimistic Locking
- • Advanced Rate Limiting & Performance Optimization
- • Database Indexing for Scale
- • Comprehensive Logging & Monitoring
- • Error Handling & Recovery
- • Health Check Endpoints
⚡ Challenges in the Task
🔄 Handling Webhooks
Webhooks can arrive out of order, be duplicated, or fail during processing. The system must handle these scenarios gracefully while maintaining data consistency and providing reliable processing.
🔒 Idempotency
Ensuring that processing the same webhook multiple times doesn't result in duplicate data or incorrect state changes. Critical for maintaining data integrity in distributed systems.
⚡ Concurrency / Race Conditions
Multiple webhooks processing simultaneously can lead to race conditions, especially when updating order statuses. The system must handle concurrent access safely without data corruption.
📋 Queue Processing & Retries
Background job processing with intelligent retry mechanisms, exponential backoff, and failure handling. Essential for reliable webhook processing at scale.
📊 Audit Trail
Complete tracking of all order status changes, webhook processing events, and system activities. Provides transparency and debugging capabilities for production systems.
📦 Out-of-Order Webhook Delivery
Webhooks may arrive in different order than they were sent, requiring intelligent handling to maintain correct order state transitions and business logic.
🚦 Rate Limitation & System Overload
High-volume webhook scenarios (20M+ records) can overwhelm the system, causing performance degradation, timeouts, and potential service unavailability. Without proper rate limiting, the system becomes vulnerable to DDoS attacks and resource exhaustion.
Specific Problems:
- • Resource Exhaustion: Unlimited requests can consume all server resources
- • Database Overload: Too many concurrent database operations
- • Memory Issues: Excessive memory usage from concurrent processing
- • Service Degradation: Slow response times affecting legitimate users
- • Security Vulnerabilities: Potential for abuse and DDoS attacks
🛠️ Solutions & Code
🔄 Webhook Handling Solution
The webhook controller implements rate limiting, duplicate detection, and immediate queuing for database transaction-based processing.
// routes/api.php - Rate limiting via middleware
Route::post('/webhooks/payments', [WebhookController::class, 'store'])
->middleware('webhook.rate.limit:1000,1');
// WebhookController.php - Clean separation of concerns
public function store(WebhookPaymentRequest $request): JsonResponse
{
$startTime = microtime(true);
// CRITICAL: Duplicate detection using cache for high performance
$validatedData = $request->validated();
$txnId = $validatedData['txn_id'];
$cacheKey = "webhook_txn:{$txnId}";
if (Cache::has($cacheKey)) {
return response()->json(['message' => 'Duplicate webhook detected'], 409);
}
// CRITICAL: Create webhook log and dispatch job with high priority
$webhookLog = WebhookLog::create([...]);
ProcessPaymentWebhookJob::dispatch($webhookLog->id)
->onQueue('webhooks-high-priority');
$processingTime = round((microtime(true) - $startTime) * 1000, 2);
return response()->json([
'message' => 'Webhook received and queued for processing',
'processing_time_ms' => $processingTime,
], 202);
}
🔒 Idempotency Solution
Multiple layers of idempotency protection using transaction IDs and database constraints.
// Check if external_txn_id already exists
if ($payload['txn_id'] && Order::withExternalTxnId($payload['txn_id'])->exists()) {
Log::info("Transaction ID {$payload['txn_id']} already processed, ignoring webhook");
$webhookLog->markAsIgnored('Transaction already processed');
return;
}
// Cache-based duplicate prevention
Cache::put($cacheKey, true, 300); // 5 minutes cache
// Database unique constraint on external_txn_id
Schema::table('orders', function (Blueprint $table) {
$table->unique('external_txn_id');
});
⚡ Concurrency Solution
Database transaction with optimistic locking strategy prevents race conditions while maintaining ACID compliance.
// CRITICAL: Use database transaction with optimistic locking
DB::transaction(function () use ($webhookLog, $payload, $startTime) {
// CRITICAL: Get order without locking (optimistic approach)
$order = Order::where('id', $payload['order_uuid'])->first();
// CRITICAL: Optimistic locking - update with version check
$updatedRows = Order::where('id', $order->id)
->where('updated_at', $order->updated_at) // Optimistic lock
->update([
'status' => $newStatus,
'external_txn_id' => $payload['txn_id'],
'amount' => $payload['amount'],
'updated_at' => now(),
]);
// CRITICAL: Check if update succeeded (optimistic locking)
if ($updatedRows === 0) {
Log::warning("Optimistic lock failed for order {$order->id} - another process modified it");
throw new \Exception('Optimistic lock failed - order was modified by another process');
}
// Create audit trail and update webhook log within same transaction
OrderEvent::create([...]);
$webhookLog->markAsProcessed($processingTime);
});
DB::transaction() for ACID compliance with optimistic locking.
The updated_at timestamp serves as a version field. If another process modified the record,
the update returns 0 rows and triggers a retry via Laravel's job retry mechanism.
📋 Queue Processing Solution
Database transaction-based processing with intelligent retry mechanism and comprehensive error handling.
class ProcessPaymentWebhookJob implements ShouldQueue
{
public int $tries = 5;
public int $timeout = 60;
/**
* Calculate the number of seconds to wait before retrying the job.
* Optimized for database transaction conflicts and deadlocks.
*/
public function backoff(): array
{
return [5, 10, 30, 60, 120]; // 5s, 10s, 30s, 1m, 2m
}
public function handle(): void
{
try {
// CRITICAL: Use database transaction with optimistic locking
DB::transaction(function () use ($webhookLog, $payload, $startTime) {
// Process webhook with optimistic locking
$order = Order::where('id', $payload['order_uuid'])->first();
$updatedRows = Order::where('id', $order->id)
->where('updated_at', $order->updated_at)
->update([...]);
if ($updatedRows === 0) {
throw new \Exception('Optimistic lock failed');
}
});
} catch (\Exception $e) {
// Handle optimistic lock failures - these should be retried
if (str_contains($e->getMessage(), 'Optimistic lock failed')) {
Log::warning("Optimistic lock failed for webhook {$this->webhookLogId}, will retry");
throw $e; // Re-throw to trigger retry mechanism
}
throw $e;
}
}
}
DB::transaction() for ACID compliance with optimistic locking.
Shorter backoff delays (5s-2m) are optimized for transaction conflicts. Laravel's job retry mechanism
automatically handles optimistic lock failures.
📊 Audit Trail Solution
Comprehensive event logging with detailed metadata for complete traceability.
// Insert new row in order_events (audit trail)
OrderEvent::create([
'order_id' => $order->id,
'from_status' => $oldStatus,
'to_status' => $newStatus,
'reason' => OrderEvent::REASON_WEBHOOK_PAYMENT,
'metadata' => [
'webhook_log_id' => $webhookLog->id,
'txn_id' => $payload['txn_id'],
'timestamp' => $payload['timestamp'],
'amount' => $payload['amount'],
'webhook_source' => $webhookLog->webhook_source,
'correlation_id' => $webhookLog->correlation_id,
],
]);
// Webhook log tracking
$webhookLog = WebhookLog::create([
'order_uuid' => $validatedData['order_uuid'],
'txn_id' => $txnId,
'raw_payload' => $validatedData,
'status' => WebhookLog::STATUS_PENDING,
'attempts' => 1,
'webhook_source' => $validatedData['webhook_source'] ?? WebhookLog::SOURCE_UNKNOWN,
'correlation_id' => $validatedData['correlation_id'] ?? null,
]);
🚦 Rate Limiting Solution
Multi-layered rate limiting strategy using Laravel's built-in rate limiter with IP-based throttling and configurable limits optimized for high-volume webhook scenarios.
1. Custom Rate Limiting Middleware
class WebhookRateLimit
{
/**
* Handle an incoming request.
*
* Rate limiting middleware specifically designed for webhook endpoints
* to handle high-pressure scenarios with 20M+ records.
*/
public function handle(Request $request, Closure $next, int $maxAttempts = 1000, int $decayMinutes = 1): Response
{
// CRITICAL: Rate limiting for high-pressure webhook scenarios
$key = 'webhook:' . $request->ip();
if (RateLimiter::tooManyAttempts($key, $maxAttempts)) {
return response()->json([
'message' => 'Too many requests',
'retry_after' => RateLimiter::availableIn($key),
'limit' => $maxAttempts,
'window' => $decayMinutes . ' minute(s)',
], 429);
}
// Increment the rate limit counter
RateLimiter::hit($key, $decayMinutes * 60);
return $next($request);
}
}
2. Route Configuration
// routes/api.php - Rate limiting via middleware
Route::post('/webhooks/payments', [WebhookController::class, 'store'])
->middleware('webhook.rate.limit:1000,1');
// Alternative: More restrictive for sensitive endpoints
Route::post('/webhooks/admin', [AdminWebhookController::class, 'store'])
->middleware('webhook.rate.limit:100,1'); // 100 requests per minute
3. Controller Integration
public function store(WebhookPaymentRequest $request): JsonResponse
{
$startTime = microtime(true);
// CRITICAL: Duplicate detection using cache for high performance
$validatedData = $request->validated();
$txnId = $validatedData['txn_id'];
$cacheKey = "webhook_txn:{$txnId}";
if (Cache::has($cacheKey)) {
return response()->json([
'message' => 'Duplicate webhook detected',
'status' => 'ignored',
'txn_id' => $txnId,
], 409);
}
// Process webhook...
$processingTime = round((microtime(true) - $startTime) * 1000, 2);
return response()->json([
'message' => 'Webhook received and queued for processing',
'processing_time_ms' => $processingTime,
], 202);
}
Rate Limiting Benefits:
- • System Protection: Prevents resource exhaustion and service degradation
- • Fair Usage: Ensures all clients get fair access to the API
- • DDoS Mitigation: Protects against malicious traffic spikes
- • Performance Optimization: Maintains consistent response times
- • Cost Control: Prevents unexpected infrastructure costs
- • Monitoring: Provides clear metrics on usage patterns
🔄 System Architecture & Flowcharts
System Architecture
Webhook Processing Flow
Optimistic Locking Strategy
Rate Limiting Flow
🌟 Advantages
🛡️ Reliability
- • Idempotency: No duplicate processing
- • Retry Logic: Automatic failure recovery
- • Transaction Safety: ACID compliance
- • Error Handling: Comprehensive exception management
- • Health Checks: System monitoring endpoints
📈 Scalability
- • Queue Processing: Asynchronous job handling
- • Rate Limiting: Prevents system overload
- • Database Indexing: Optimized queries
- • Cache Strategy: Reduced database load
- • Optimistic Locking: No blocking operations
🔧 Maintainability
- • Clean Architecture: Separation of concerns
- • Comprehensive Logging: Easy debugging
- • Type Safety: Strong typing throughout
- • Documentation: Well-documented code
- • Testing: Comprehensive test coverage
👁️ Observability
- • Audit Trail: Complete event history
- • Performance Metrics: Processing time tracking
- • Structured Logging: Searchable log entries
- • Webhook Tracking: Full request/response logs
- • Status Monitoring: Real-time system health
📝 Example Usage
Webhook Payload Example
{
"order_uuid": "550e8400-e29b-41d4-a716-446655440000",
"txn_id": "TXN_123456789",
"status": "paid",
"amount": 99.99,
"timestamp": "2024-01-15T10:30:00Z",
"webhook_source": "payment_gateway",
"correlation_id": "CORR_789",
"metadata": {
"payment_method": "credit_card",
"card_last_four": "1234",
"currency": "USD",
"customer_id": "CUST_456"
}
}
API Call Example (CURL)
curl -X POST http://localhost:8000/api/webhooks/payment \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_API_TOKEN" \
-d '{
"order_uuid": "550e8400-e29b-41d4-a716-446655440000",
"txn_id": "TXN_123456789",
"status": "paid",
"amount": 99.99,
"timestamp": "2024-01-15T10:30:00Z",
"webhook_source": "payment_gateway",
"correlation_id": "CORR_789",
"metadata": {
"payment_method": "credit_card",
"card_last_four": "1234",
"currency": "USD",
"customer_id": "CUST_456"
}
}'
Success Response
{
"message": "Webhook received and queued for processing",
"webhook_log_id": 12345,
"txn_id": "TXN_123456789",
"status": "queued",
"processing_time_ms": 15.23
}
🚀 Quick Start
1. Install Dependencies
composer install
2. Environment Setup
cp .env.example .env
php artisan key:generate
Configure your database connection and other environment variables in the .env file.
3. Database Setup
php artisan migrate:fresh --seed
This will create all necessary tables and populate them with sample data.
4. Start Queue Worker
php artisan queue:work --queue=webhooks-high-priority
Start the queue worker to process webhook jobs. Run this in a separate terminal.
5. Run Tests
php artisan test
Verify that all tests pass to ensure the system is working correctly.
6. Start Development Server
php artisan serve
The application will be available at http://localhost:8000