📦 EqualifyEverything / benchmarks-ai-alt

📄 functional-images.jsonl · 13 lines
1
2
3
4
5
6
7
8
9
10
11
12
13{"id":"fi-0001","status":"rejected","round_added":1,"category":1,"subtype":"linked-standalone-logo","page_url":"https://www.w3.org/WAI/tutorials/images/functional/","domain":"w3.org","image_url":"https://www.w3.org/WAI/content-images/tutorials/images/w3c.png","implementation":"img","element_role":"link","element_html":"<a href=\"https://www.w3.org/\"><img src=\"/WAI/content-images/tutorials/images/w3c.png\" alt=\"W3C home\" /></a>","surrounding_text":"","destination":"https://www.w3.org/","observed_alt":"W3C home","observed_alt_verdict":"correct","gold_alt":"W3C home","gold_alt_rationale":"Names the destination (W3C homepage) rather than describing the visual. The logo is the sole content of the link, so it must carry the accessible name. Rejected 'W3C Logo' because it describes appearance rather than function, per the action-over-description criterion.","gold_alt_passes":[{"author":"pass-a","alt":"W3C home","rationale":"Sole image in link to w3.org root; must name destination. Logo visual is irrelevant; 'home' identifies where it goes."},{"author":"pass-b","alt":"W3C home","rationale":"Link destination is the W3C homepage; label the outcome not the glyph. No adjacent text, so cannot be empty."}],"adjudication":null,"difficulty":"standard","dual_purpose":false,"leakage_check":"Observed alt matches gold exactly, making the answer recoverable from the site's own alt attribute. This leaks.","leaky":true,"retrieved":"2026-08-27","provenance_note":"Public WAI tutorial page, no login; image referenced by URL only, not redistributed."}
{"id":"fi-0002","status":"accepted","round_added":1,"category":1,"subtype":"linked-standalone-logo","page_url":"https://www.w3.org/WAI/tutorials/images/functional/","domain":"w3.org","image_url":"https://www.w3.org/WAI/content-images/tutorials/images/w3c.png","implementation":"img","element_role":"link","element_html":"<a href=\"https://www.w3.org/\" style=\"display:inline-block;\"><img src=\"/WAI/content-images/tutorials/images/w3c.png\" alt=\"\" style=\"vertical-align: middle; margin-right: 1em;\" />W3C Home</a>","surrounding_text":"W3C Home","destination":"https://www.w3.org/","observed_alt":"","observed_alt_verdict":"empty-appropriate","gold_alt":"","gold_alt_rationale":"The link contains visible text 'W3C Home' that already names the destination. Adding alt text to the image would cause a screen reader to announce the destination twice. Per the redundancy criterion in the WAI decision tree, the correct answer is an empty alt attribute. Rejected 'W3C home' because it duplicates the adjacent text node.","gold_alt_passes":[{"author":"pass-a","alt":"","rationale":"Adjacent text 'W3C Home' inside the same link already names the destination; image is redundant, per redundancy-elimination criterion."},{"author":"pass-b","alt":"","rationale":"Text node in the same anchor provides the accessible name. Adding alt text would repeat the link name. Empty alt is correct per W3C decision tree."}],"adjudication":null,"difficulty":"standard","dual_purpose":false,"leakage_check":"File name w3c.png and URL path reveal nothing about whether alt should be empty; requires seeing the surrounding text node inside the link.","leaky":false,"retrieved":"2026-08-27","provenance_note":"Public WAI tutorial page, no login; cited by URL only."}
{"id":"fi-0003","status":"rejected","round_added":1,"category":1,"subtype":"standalone-navigational-link","page_url":"https://www.w3.org/WAI/tutorials/images/functional/","domain":"w3.org","image_url":"https://www.w3.org/WAI/content-images/tutorials/images/print.png","implementation":"img","element_role":"link","element_html":"<a href=\"javascript:print()\"><img src=\"/WAI/content-images/tutorials/images/print.png\" alt=\"Print this page\" /></a>","surrounding_text":"","destination":"Opens the browser print dialog","observed_alt":"Print this page","observed_alt_verdict":"correct","gold_alt":"Print this page","gold_alt_rationale":"The image is the sole content of a link that triggers the print dialog. It must name the action, not describe the glyph. 'Print this page' states the outcome. Rejected 'Printer icon' because it describes appearance without communicating function. The addition of 'this page' adds useful scope given the page context.","gold_alt_passes":[{"author":"pass-a","alt":"Print this page","rationale":"Sole image in action link; must name the action. 'Print this page' labels the outcome per action-over-description."},{"author":"pass-b","alt":"Print this page","rationale":"Icon triggers print dialog; function over visual. Rejected 'Print' alone because 'this page' clarifies scope at minimal cost."}],"adjudication":null,"difficulty":"standard","dual_purpose":false,"leakage_check":"File name print.png hints at 'print' but not the full gold standard 'Print this page'. Observed alt matches gold exactly, making it recoverable from the site's own attribute.","leaky":true,"retrieved":"2026-08-27","provenance_note":"Public WAI tutorial page, no login; cited by URL only."}
{"id":"fi-0004","status":"rejected","round_added":1,"category":2,"subtype":"form-control-or-image-button","page_url":"https://www.w3.org/WAI/tutorials/images/functional/","domain":"w3.org","image_url":"https://www.w3.org/WAI/content-images/tutorials/images/searchbutton.png","implementation":"input-image","element_role":"input-image","element_html":"<input name=\"submit\" src=\"/WAI/content-images/tutorials/images/searchbutton.png\" alt=\"Search\" type=\"image\" style=\"vertical-align: middle; display:inline-block; border: none;\" />","surrounding_text":"Search:","destination":"Submits the search form","observed_alt":"Search","observed_alt_verdict":"correct","gold_alt":"Search","gold_alt_rationale":"The input type=image submits the search form. The adjacent label reads 'Search:' naming the field, and the button's function is to execute that search. 'Search' names the action concisely. Rejected 'Submit' because it is generic and 'Magnifying glass' because it describes the visual. The label and button name the same concept but serve different roles (field vs. action), so the button's alt is not redundant.","gold_alt_passes":[{"author":"pass-a","alt":"Search","rationale":"Image button submits a search query; label the action per action-over-description. 'Magnifying glass' rejected as visual description."},{"author":"pass-b","alt":"Search","rationale":"Function is form submission for search. Names the outcome, not the glyph. Brief per character guidance."}],"adjudication":null,"difficulty":"standard","dual_purpose":false,"leakage_check":"File name searchbutton.png contains 'search', and observed alt is 'Search'. Gold is recoverable from either. Item leaks.","leaky":true,"retrieved":"2026-08-27","provenance_note":"Public WAI tutorial page, no login; cited by URL only."}
{"id":"fi-0005","status":"rejected","round_added":1,"category":2,"subtype":"form-control-or-image-button","page_url":"https://www.gutenberg.org/","domain":"gutenberg.org","image_url":"https://www.gutenberg.org/pics/en_US.gif","implementation":"input-image","element_role":"input-image","element_html":"<input class=\"donbtn\" type=\"image\" src=\"/pics/en_US.gif\" name=\"submit\" alt=\"Donate via PayPal\">","surrounding_text":"Donate","destination":"Submits PayPal donation form (opens PayPal in new window)","observed_alt":"Donate via PayPal","observed_alt_verdict":"correct","gold_alt":"Donate via PayPal","gold_alt_rationale":"The input type=image submits a PayPal donation form. 'Donate via PayPal' names both the action and the specific payment processor, which is information the user needs before activating. There is an adjacent text link that says 'Donate' but it is a separate control pointing to a different URL (/donate/), so the image button is not redundant. Rejected 'Submit' as too generic and 'PayPal button' as describing the visual rather than the function.","gold_alt_passes":[{"author":"pass-a","alt":"Donate via PayPal","rationale":"Image button submits a PayPal form; names the action and processor. Separate 'Donate' link goes elsewhere, so not redundant."},{"author":"pass-b","alt":"Donate via PayPal","rationale":"Function is submitting donation via PayPal specifically. Must distinguish from adjacent /donate/ link. Labels outcome and method."}],"adjudication":null,"difficulty":"ambiguous","dual_purpose":true,"leakage_check":"File name en_US.gif reveals nothing about donation. Observed alt matches gold, making it recoverable from the site's own attribute.","leaky":true,"retrieved":"2026-08-27","provenance_note":"Public Project Gutenberg homepage, no login; cited by URL only.","notes":"The image file en_US.gif appears to be a PayPal-provided button graphic for the US English locale. Adjacent text link 'Donate' goes to /donate/ which is a different destination."}
{"id":"fi-0006","status":"accepted","round_added":1,"category":1,"subtype":"linked-standalone-logo","page_url":"https://www.gutenberg.org/","domain":"gutenberg.org","image_url":"https://www.gutenberg.org/gutenberg/pg-logo-new.jpg","implementation":"img","element_role":"link","element_html":"<a id=\"main_logo\" href=\"/\" class=\"no-hover\"><img src=\"/gutenberg/pg-logo-new.jpg\" alt=\"Project Gutenberg\" draggable=\"false\" ></a>","surrounding_text":"","destination":"https://www.gutenberg.org/","observed_alt":"Project Gutenberg","observed_alt_verdict":"wrong","gold_alt":"Project Gutenberg home","gold_alt_rationale":"The logo is the sole content of a link to the site root. It must name the destination. 'Project Gutenberg' names the organization but does not communicate that activating it navigates home. Adding 'home' follows the W3C pattern of destination-over-label for linked logos. Rejected 'Project Gutenberg' alone because a user hearing only the org name cannot distinguish this from a non-navigational logo. However, this is a genuinely ambiguous case: 'Project Gutenberg' alone is defensible because screen readers announce 'link' role and the href is '/', making 'home' inferable.","gold_alt_passes":[{"author":"pass-a","alt":"Project Gutenberg home","rationale":"Linked logo to site root; destination-over-label says name the place you go. Adding 'home' disambiguates from informative logo use."},{"author":"pass-b","alt":"Project Gutenberg","rationale":"Logo links to root; the org name is sufficient because screen readers announce the link role and '/' is understood as home. Adding 'home' is helpful but not strictly required."}],"adjudication":"Pass A wins. The W3C functional images tutorial uses 'W3C home' as the exemplar for exactly this pattern. While pass B's reasoning is valid, the benchmark should prefer the more explicit label that works without relying on the user inferring destination from URL structure.","difficulty":"ambiguous","dual_purpose":false,"leakage_check":"File name pg-logo-new.jpg does not reveal 'home'. Observed alt 'Project Gutenberg' differs from gold. Requires understanding link context to decide on 'home' suffix.","leaky":false,"retrieved":"2026-08-27","provenance_note":"Public Project Gutenberg homepage, no login; cited by URL only."}
{"id":"fi-0007","status":"needs-revision","round_added":1,"category":1,"subtype":"linked-standalone-logo","page_url":"https://www.loc.gov/","domain":"loc.gov","image_url":null,"implementation":"inline-svg","element_role":"link","element_html":"<a href=\"/\"><svg xmlns=\"http://www.w3.org/2000/svg\" width=\"363\" height=\"90\" viewBox=\"0 0 363 90\"><g fill=\"none\"><path d=\"m15.658 0 .108 75.361h9.644l-.108 14.205H.488V0z\" fill=\"#000\" class=\"lib\"/></g></svg><span>Library of Congress</span></a>","surrounding_text":"Library of Congress","destination":"https://www.loc.gov/","observed_alt":null,"observed_alt_verdict":"missing","gold_alt":"","gold_alt_rationale":"The link contains a visible text span 'Library of Congress' that serves as the accessible name for the entire link. The SVG is a visual rendering of the same wordmark. Adding alt text to the SVG would cause duplication. Per the redundancy criterion, the image should be hidden from assistive technology or carry an empty alt equivalent. Rejected 'Library of Congress home' because the text span already provides this information.","gold_alt_passes":[{"author":"pass-a","alt":"","rationale":"Adjacent span 'Library of Congress' inside the same link names the destination. SVG is redundant visual; empty alt prevents duplication per redundancy criterion."},{"author":"pass-b","alt":"","rationale":"Text node in the anchor provides the link name. The SVG wordmark repeats it visually. Correct answer is to hide it from AT, equivalent to empty alt."}],"adjudication":null,"difficulty":"standard","dual_purpose":false,"leakage_check":"The SVG has no alt attribute or aria-label in the served HTML. File path is inline. Determining that empty is correct requires seeing the adjacent text span, which is not in the image URL or file name.","leaky":false,"retrieved":"2026-08-27","provenance_note":"Public Library of Congress homepage, no login; cited by URL only.","notes":"The element_html is abbreviated from the full SVG path data for readability but the key structure is preserved: the link contains both the SVG and a text span. The full SVG is a wordmark rendering 'LIBRARY OF CONGRESS' in paths."}
{"id":"fi-0008","status":"needs-revision","round_added":1,"category":1,"subtype":"linked-standalone-logo","page_url":"https://www.gov.uk/","domain":"gov.uk","image_url":null,"implementation":"inline-svg","element_role":"link","element_html":"<a class=\"govuk-header__homepage-link\" id=\"logo\" aria-label=\"Go to the GOV.UK homepage\" href=\"/\"><svg xmlns=\"http://www.w3.org/2000/svg\" focusable=\"false\" role=\"img\" viewBox=\"0 0 324 60\" height=\"30\" width=\"162\" fill=\"currentcolor\" class=\"govuk-header__logotype\" aria-label=\"GOV.UK\"><title>GOV.UK</title></svg></a>","surrounding_text":"","destination":"https://www.gov.uk/","observed_alt":"GOV.UK","observed_alt_verdict":"wrong","gold_alt":"GOV.UK homepage","gold_alt_rationale":"The SVG has aria-label 'GOV.UK' and a title element, but the link itself carries aria-label 'Go to the GOV.UK homepage'. The link's aria-label overrides the SVG's in terms of what a screen reader announces for the link. The gold standard for the image inside this link should communicate that it links to the homepage. The observed SVG alt ('GOV.UK' via title) names the organization but not the destination. Following destination-over-label, adding 'homepage' is correct. Rejected 'Go to the GOV.UK homepage' because it contains the redundant starter 'Go to'. Rejected 'GOV.UK' alone as it does not name the destination.","gold_alt_passes":[{"author":"pass-a","alt":"GOV.UK homepage","rationale":"Logo links to site root; destination-over-label applies. 'homepage' identifies where it goes. Rejected 'Go to' prefix as redundant starter."},{"author":"pass-b","alt":"","rationale":"The link itself has aria-label 'Go to the GOV.UK homepage' which provides the accessible name for the entire interactive element. The SVG inside is decorative relative to the link's own label and should carry no additional alt to avoid double-announcement."}],"adjudication":"Pass B wins. The link element carries its own aria-label that fully describes the destination. In this specific implementation, the SVG's accessible name is overridden by the link's aria-label at the accessibility tree level. The correct alt for the image within this already-labelled link is empty, to prevent the screen reader from announcing both the link label and the image label. This is the same redundancy pattern as fi-0002 but implemented via aria-label rather than visible text.","difficulty":"ambiguous","dual_purpose":false,"leakage_check":"The SVG title 'GOV.UK' and aria-label 'GOV.UK' are visible in markup but the gold standard (empty) is not recoverable from them. Requires understanding the link's own aria-label creates the accessible name.","leaky":false,"retrieved":"2026-08-27","provenance_note":"Public GOV.UK homepage, no login; cited by URL only.","notes":"Complex accessibility pattern: the link has aria-label='Go to the GOV.UK homepage' which provides the accessible name for the entire control. The SVG's own labelling is nested inside an already-named interactive element."}
{"id":"fi-0009","status":"needs-revision","round_added":1,"category":1,"subtype":"standalone-navigational-link","page_url":"https://www.harvard.edu/","domain":"harvard.edu","image_url":"https://www.harvard.edu/wp-content/uploads/2026/04/Instagram.svg","implementation":"img","element_role":"link","element_html":"<a href=\"https://www.instagram.com/harvard/\" class=\"social-follow__anchor\" rel=\"me noopener\" title=\"Instagram\"><span>Instagram</span><img src=\"https://www.harvard.edu/wp-content/uploads/2026/04/Instagram.svg\" alt=\"Instagram\" aria-hidden=\"true\" /></a>","surrounding_text":"Instagram","destination":"https://www.instagram.com/harvard/","observed_alt":"Instagram","observed_alt_verdict":"empty-appropriate","gold_alt":"","gold_alt_rationale":"The link contains a visible text span 'Instagram' that names the destination. The image is explicitly marked aria-hidden='true' by the site, correctly hiding it from assistive technology. The gold standard for the image is empty alt because the text span already provides the accessible name for the link. Adding alt text would be redundant even if aria-hidden were removed. Rejected 'Instagram' because it duplicates the text node in the same link, per the redundancy criterion.","gold_alt_passes":[{"author":"pass-a","alt":"","rationale":"Text span 'Instagram' inside the same link names the destination. Image is correctly aria-hidden. Even without aria-hidden, empty alt is correct to prevent redundancy."},{"author":"pass-b","alt":"","rationale":"Adjacent text in same anchor provides the link name. Redundancy criterion dictates empty alt. The site correctly uses aria-hidden='true' on the decorative icon."}],"adjudication":null,"difficulty":"standard","dual_purpose":false,"leakage_check":"File name Instagram.svg and observed alt 'Instagram' both reveal the platform name, but the gold standard is empty string, which is not recoverable from file name or observed alt alone. Requires seeing the text span to know redundancy applies.","leaky":false,"retrieved":"2026-08-27","provenance_note":"Public Harvard University homepage, no login; cited by URL only."}
{"id":"fi-0010","status":"accepted","round_added":1,"category":4,"subtype":"linked-complex-graphic-or-image-map","page_url":"https://geology.com/world/world-map.shtml","domain":"geology.com","image_url":"https://geology.com/world/world-map.gif","implementation":"img","element_role":"area","element_html":"<img src=\"https://geology.com/world/world-map.gif\" alt=\"world map\" width=\"1200\" height=\"715\" vspace=\"8\" usemap=\"#Map\" class=\"image\">","surrounding_text":"World Map - Political - Click a Country","destination":"Multiple country-specific pages via area elements","observed_alt":"world map","observed_alt_verdict":"wrong","gold_alt":"Political world map with clickable countries","gold_alt_rationale":"This is an image map where the img element needs alt text that describes the overall graphic and its interactive nature. The observed 'world map' is too brief and fails to communicate that the regions are clickable or what clicking does. The gold standard adds 'political' (matching the page heading) and 'with clickable countries' to convey the interactive function. Rejected 'world map' alone as insufficient for the image map context. Rejected a full listing of countries as too verbose for the img alt (that belongs on the individual area elements).","gold_alt_passes":[{"author":"pass-a","alt":"Political world map with clickable countries","rationale":"Image map needs overall context label. Must communicate both what the graphic shows and that regions are interactive. 'Political' from page heading; 'clickable countries' names the interaction model."},{"author":"pass-b","alt":"World map with clickable regions linking to country information","rationale":"Names the graphic type and the function of the areas. 'linking to country information' describes what clicking does. More specific than just 'clickable'."}],"adjudication":"Pass A wins. Both are defensible, but pass B is slightly verbose and 'linking to country information' adds words without proportional clarity gain. The individual area elements carry their own alt text naming each country. The img alt should provide overall context without trying to replicate the area labels.","difficulty":"ambiguous","dual_purpose":true,"leakage_check":"File name world-map.gif and observed alt 'world map' hint at the content but not the interactive nature or 'political' qualifier. Gold requires understanding the usemap context.","leaky":false,"retrieved":"2026-08-27","provenance_note":"Public geology.com page, no login required; robots.txt allows all. Cited by URL only.","notes":"The map element contains dozens of area elements for individual countries, each with their own alt text like 'Canada map', 'United States map', etc. This item records the overall image; individual areas could be separate items."}
{"id":"fi-0011","status":"needs-revision","round_added":1,"category":1,"subtype":"standalone-navigational-link","page_url":"https://www.nps.gov/subjects/digital/nps-apps.htm","domain":"nps.gov","image_url":"https://www.nps.gov/subjects/digital/images/apple-badge_1.png","implementation":"img","element_role":"link","element_html":"<a href=\"https://apps.apple.com/us/app/national-park-service/id1549226484\"><img alt=\"Download from the App Store\" title=\"Download from the App Store\" src=\"/subjects/digital/images/apple-badge_1.png?maxwidth=1300&maxheight=1300&autorotate=false&format=webp\"></a>","surrounding_text":"","destination":"https://apps.apple.com/us/app/national-park-service/id1549226484","observed_alt":"Download from the App Store","observed_alt_verdict":"correct","gold_alt":"Download NPS App on the App Store","gold_alt_rationale":"The image is an Apple App Store badge inside a link to the NPS app listing. The observed alt 'Download from the App Store' names the action and store but not which app. Adding 'NPS App' identifies what will be downloaded, which a user needs to make an informed decision. This is a dual-purpose item: the badge is both a functional link and carries informative content (the Apple brand, the store identity). Rejected 'Download from the App Store' as slightly underspecified. Rejected 'Apple App Store badge' as describing the visual rather than the function.","gold_alt_passes":[{"author":"pass-a","alt":"Download NPS App on the App Store","rationale":"Badge links to a specific app listing. Must name both the action (download) and what is being downloaded (NPS App). Store identity adds useful context about the platform."},{"author":"pass-b","alt":"Download from the App Store","rationale":"Names the action and destination store. The specific app is clear from page context (NPS Apps page). Adding the app name is helpful but the page context makes it inferable."}],"adjudication":"Pass A wins. While page context does make the app identity inferable, the alt text should stand on its own for a user navigating by links list, where page context may not be visible. Naming the specific app follows the principle of providing equivalent information.","difficulty":"ambiguous","dual_purpose":true,"leakage_check":"File name apple-badge_1.png hints at Apple but not the full gold standard. Observed alt 'Download from the App Store' differs from gold. Not fully recoverable without context.","leaky":false,"retrieved":"2026-08-27","provenance_note":"Public NPS page, no login; robots.txt allows /subjects/. Cited by URL only."}
{"id":"fi-0012","status":"rejected","round_added":1,"category":2,"subtype":"action-or-toggle-icon","page_url":"https://www.w3.org/WAI/tutorials/images/functional/","domain":"w3.org","image_url":"https://www.w3.org/WAI/assets/images/icons.svg","implementation":"sprite","element_role":"button","element_html":"<button class=\"button button-icon button-nobg button-noborder\"><span><svg focusable=\"false\" aria-label=\"Submit Search\" class=\"icon-search \"><use xlink:href=\"/WAI/assets/images/icons.svg#icon-search\"></use></svg></span></button>","surrounding_text":"","destination":"Submits the WAI site search form","observed_alt":"Submit Search","observed_alt_verdict":"correct","gold_alt":"Search","gold_alt_rationale":"The SVG carries aria-label 'Submit Search' which names the action. However, 'Submit Search' contains the redundant word 'Submit' given that it is already inside a button element (the role communicates that activating it will submit). The gold standard 'Search' is more concise and follows the brevity criterion. The button is inside a search form, so 'Search' fully names the action. Rejected 'Submit Search' because 'Submit' is a redundant starter equivalent to 'button for'. Rejected 'Magnifying glass' as visual description.","gold_alt_passes":[{"author":"pass-a","alt":"Search","rationale":"Button in search form; 'Search' names the action concisely. 'Submit' is redundant given the button role. Brevity criterion applies."},{"author":"pass-b","alt":"Submit Search","rationale":"Names the action directly. 'Submit' clarifies it executes the search rather than opening a search field. Not technically a redundant starter since it names the specific action of submitting."}],"adjudication":"Pass A wins. 'Submit' before 'Search' is functionally equivalent to 'button for search' because the button role already communicates activation. A screen reader would announce 'Submit Search, button' which is redundant. 'Search, button' is cleaner and equally informative.","difficulty":"ambiguous","dual_purpose":false,"leakage_check":"Sprite reference is icons.svg#icon-search. The word 'search' appears in the sprite ID and in the observed aria-label. Gold 'Search' is recoverable from the sprite symbol name.","leaky":true,"retrieved":"2026-08-27","provenance_note":"Public WAI tutorial page, no login; cited by URL only.","notes":"Implementation is an SVG sprite reference. The icon set is WAI's own icon system. The button sits inside a search form in the site's meta navigation."}