1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194# Scanning Services
Equalify's accessibility scanning is performed by a collection of microservices that work together to scan HTML pages and PDF documents at scale.
## Overview
The scanning architecture uses AWS SQS FIFO queues to distribute work across scanner Lambdas, ensuring reliable delivery and ordered processing per audit.
```
┌─────────────┐ ┌──────────────┐ ┌─────────────┐ ┌───────────┐
│ Backend │────▶│ SQS Router │────▶│ SQS Queue │────▶│ Scanner │
│ API │ │ Lambda │ │ (FIFO) │ │ Lambda │
└─────────────┘ └──────────────┘ └─────────────┘ └───────────┘
│
▼
┌───────────┐
│ Webhook │
│ (Backend) │
└───────────┘
```
## Service Components
### SQS Router (`aws-lambda-scan-sqs-router`)
**Purpose**: Receives scan requests and routes URLs to appropriate SQS queues based on content type.
**Input Schema** (Zod validated):
```typescript
const scansSchema = z.object({
urls: z.array(
z.object({
auditId: z.string(),
scanId: z.string(),
urlId: z.string(),
url: z.string(),
type: z.string(), // "html" or "pdf"
isStaging: z.boolean().optional(),
})
),
});
```
**Behavior**:
1. Parse and validate incoming URL list
2. Separate URLs by type (HTML vs PDF)
3. Batch URLs into groups of 10 (SQS limit)
4. Send batches to appropriate FIFO queues
**Queue Configuration**:
- HTML Queue: `scanHtml.fifo`
- PDF Queue: `scanPdf.fifo`
- Message Group ID: `auditId` (ensures ordered processing per audit)
- Deduplication ID: `urlId` (prevents duplicate scans)
### HTML Scanner (`aws-lambda-scan-html`)
**Purpose**: Scans web pages for accessibility issues using a headless Chromium browser and axe-core.
**Key Features**:
- Uses AWS Lambda Powertools for metrics and logging
- Processes SQS records with partial batch failure handling
- 2-minute timeout per scan to prevent Lambda hangs
- Converts axe-core results to EqualifyV2 format
**Scan Flow**:
1. Receive SQS message with URL details
2. Launch headless Chromium browser
3. Navigate to URL and wait for page load
4. Inject and run axe-core accessibility tests
5. Convert results to EqualifyV2 format
6. POST results to webhook endpoint
**Error Handling**:
- Timeout errors: Logged and reported as failures
- Network errors: Caught and reported with error type
- Partial failures: Uses AWS Powertools batch processor
### PDF Scanner (`aws-lambda-scan-pdf`)
**Purpose**: Orchestrates PDF accessibility scanning using veraPDF.
**Architecture**:
- TypeScript Lambda receives SQS message
- Invokes Java-based veraPDF Lambda
- Converts veraPDF results to EqualifyV2 format
- Reports results via webhook
### veraPDF Interface (`aws-lambda-verapdf-interface`)
**Purpose**: Java Lambda that performs actual PDF scanning using veraPDF library.
**Features**:
- Native Java Lambda for performance
- Downloads PDF from URL
- Executes veraPDF accessibility checks
- Returns raw JSON results
## Result Format (EqualifyV2)
All scanners convert their results to a unified format:
```typescript
interface StreamResults {
status: string; // "complete" or "failed"
auditId: string;
scanId: string;
urlId: string;
url?: string;
blockers: Blocker[];
date: string;
message: string;
}
interface Blocker {
source: string; // "axe-core" | "editoria11y" | "pdf-scan"
test: string; // Rule ID (e.g., "color-contrast")
tags?: string[]; // WCAG tags (e.g., ["wcag2aa", "wcag143"])
description: string; // Human-readable rule description
summary: string; // Specific failure details
node: string | null; // HTML snippet or null for PDFs
}
```
## Axe-Core Conversion
The `AxeToEqualify2` converter transforms axe-core results:
```typescript
function convertToEqualifyV2(axeResult: AxeResults, job: any): StreamResults {
const blockers: Blocker[] = [];
// Process violations and incomplete results
[axeResult.incomplete, axeResult.violations].forEach(results => {
results?.forEach(rule => {
rule.nodes?.forEach(node => {
blockers.push({
source: "axe-core",
test: rule.id,
tags: rule.tags,
description: `${rule.description}. ${rule.help}`,
summary: node.failureSummary ?? "",
node: node.html
});
});
});
});
return { auditId, scanId, urlId, url, blockers, status: "complete", ... };
}
```
## Webhook Integration
Results are sent to the backend API webhook:
| Environment | Endpoint |
|-------------|----------|
| Production | `https://api.equalifyapp.com/public/scanWebhook` |
| Staging | `https://api-staging.equalifyapp.com/public/scanWebhook` |
**Webhook Payload**:
```json
{
"auditId": "uuid",
"scanId": "uuid",
"urlId": "uuid",
"url": "https://example.com",
"status": "complete",
"blockers": [...]
}
```
## Metrics & Monitoring
Using AWS Lambda Powertools:
- `scansStarted`: Count of scans initiated
- `ScanDuration`: Time to complete each scan (milliseconds)
- Cold start metrics captured automatically
## Adding New Scanners
To add a new scanner type:
1. Create a new service in `services/`
2. Implement SQS message consumption
3. Create a converter in `shared/convertors/`
4. Add queue routing in `aws-lambda-scan-sqs-router`
5. Register the new type in the scan schema
---
*For database schema details, see the Database Guide.*