Crawler Report for www.scrapingbee.com

Summary

Website Quality Score

7.5 Good
Performance
10.0
SEO
8.6
Security
7.0
Accessibility
5.0
Best Practices
6.7
  • ⛔ Skipped URLs - 10 skipped URLs found.
  • ⛔ Security - 22 pages(s) with critical finding(s).
  • ⚠️ Redirects - 7 redirects found.
  • ⚠️ The description '' exceeds the allowed 10% duplicity. 100% of pages have this same description.
  • ⚠️ 11 page(s) do not support Brotli compression.
  • ⚠️ No WebP image found on the website.
  • ⚠️ No AVIF image found on the website.
  • ⚠️ 11 page(s) with missing quotes on attributes.
  • ⚠️ 1 page(s) with non-clickable (non-interactive) phone numbers.
  • ⚠️ 10 page(s) without image alt attributes.
  • ⚠️ 11 page(s) without form labels.
  • ⚠️ 11 page(s) without aria labels.
  • ⚠️ 11 page(s) without role attributes.
  • ⏩ Loaded robots.txt for domain 'www.scrapingbee.com': status code 200, size 72 B and took 171 ms.
  • ⏩ External URLs - 10 external URL(s) found.
  • ✅ 404 OK - all pages exists, no non-existent pages found.
  • ✅ SSL/TLS certificate is valid until May 19 05:04:37 2026 GMT. Issued by C = US, O = Google Trust Services, CN = WE1. Subject is CN = www.scrapingbee.com.
  • ✅ SSL/TLS certificate issued by 'C = US, O = Google Trust Services, CN = WE1'.
  • ✅ Performance OK - all non-media URLs are faster than 3 seconds.
  • ✅ HTTP headers - found 15 unique headers.
  • ✅ All 11 unique title(s) are within the allowed 10% duplicity. Highest duplicity title has 9%.
  • ✅ All pages have inline SVGs smaller than 5120 bytes.
  • ✅ All pages have inline SVGs with less than 5 duplicates.
  • ✅ All pages have valid or none inline SVGs.
  • ✅ All pages without multiple <h1> headings.
  • ✅ All pages have <h1> heading.
  • ✅ All pages have heading structure without skipped levels.
  • ✅ All pages have DOM depth less than 30.
  • ✅ All pages have valid HTML.
  • ✅ All pages have lang attribute.
  • ✅ DNS IPv4 OK: domain www.scrapingbee.com resolved to scrapingbee-landing-page.pages.dev., 188.114.96.3, 188.114.97.3 (DNS server: 127.0.0.53).
  • ✅ DNS IPv6 OK: domain www.scrapingbee.com resolved to scrapingbee-landing-page.pages.dev., 2a06:98c1:3121::3, 2a06:98c1:3120::3 (DNS server: 127.0.0.53).
  • 📌 DNS Aliases: IP(s) for domain www.scrapingbee.com were resolved by CNAME chain www.scrapingbee.com > scrapingbee-landing-page.pages.dev.

Visited URLs

Found 18 row(s).
URLStatusTypeTime (s)SizeCache
/documentation308 Redirect27 ms 95 BNone
/documentation/200 HTML60 ms 624 kB0 s
/documentation/data-extraction/200 HTML57 ms 174 kB0 s
/documentation/proxy-mode/200 HTML55 ms 150 kB0 s
/documentation/chatgpt308 Redirect27 ms 111 BNone
/documentation/google-api308 Redirect30 ms 117 BNone
/documentation/fast-search308 Redirect29 ms 119 BNone
/documentation/youtube308 Redirect26 ms 111 BNone
/documentation/walmart308 Redirect26 ms 111 BNone
/documentation/js-scenario/200 HTML60 ms 156 kB0 s
/documentation/country_codes/200 HTML50 ms 162 kB0 s
/documentation/amazon308 Redirect26 ms 109 BNone
/documentation/chatgpt/200 HTML71 ms 193 kB0 s
/documentation/google-api/200 HTML54 ms 224 kB0 s
/documentation/fast-search/200 HTML97 ms 214 kB0 s
/documentation/youtube/200 HTML57 ms 566 kB0 s
/documentation/walmart/200 HTML70 ms 474 kB0 s
/documentation/amazon/200 HTML96 ms 557 kB0 s
No rows found, please edit your search term.

Best practices

Found 12 row(s).
Analysis nameOKNoticeWarningCritical
Non-clickable phone numbers0010
Missing quotes on attributes002160
Invalid inline SVGs8000
Duplicate inline SVGs (> 5 and > 1024 B)8000
Large inline SVGs (> 5120 B)8000
DOM depth (> 30)11000
Heading structure22000
Title uniqueness (> 10%)11000
Description uniqueness (> 10%)0010
Brotli support00110
WebP support0010
AVIF support0010
No rows found, please edit your search term.

Large inline SVGs

No problems found.


Duplicate inline SVGs

No problems found.


Invalid inline SVGs

No problems found.


Missing quotes on attributes

Found 200 row(s).
SeverityOccursDetailAffected URLs (max 5)
warning22The attribute 'src' has a value '/images/logo.svg' not enclosed in quotes in tag <img ***>URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value 'https://dashboard.scrapingbee.com/account/register' not enclosed in quotes in tag <a class="px-* py-* shrink-* !font-* text-* rounded transition-* bg-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value 'https://fonts.googleapis.com' not enclosed in quotes in tag <link>URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'src' has a value '/scripts/hljs.min.js' not enclosed in quotes in tag <script>URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '#' not enclosed in quotes in tag <a class="flex gap-* items-* px-* py-* rounded transition-* lg:px-* hover:opacity-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/youtube' not enclosed in quotes in tag <a class="block py-* text-* hover:underline">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value 'https://fonts.gstatic.com' not enclosed in quotes in tag <link>URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/' not enclosed in quotes in tag <a class="hidden items-* lg:flex">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value 'https://dashboard.scrapingbee.com/account/login' not enclosed in quotes in tag <a class="px-* py-* rounded transition-* shrink-* xl:px-* hover:opacity-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value 'https://www.scrapingbee.com/index.xml' not enclosed in quotes in tag <link ***>URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/walmart' not enclosed in quotes in tag <a class="block py-* text-* hover:underline">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/tutorials/' not enclosed in quotes in tag <a class="block py-* text-* hover:underline">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value 'https://dashboard.scrapingbee.com/account/login' not enclosed in quotes in tag <a class="block px-* py-* w-* text-* text-* transition-* text-* hover:bg-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value 'https://help.scrapingbee.com/en/' not enclosed in quotes in tag <a class="block py-* text-* hover:underline">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/tutorials/' not enclosed in quotes in tag <a class="block px-* py-* w-* text-* transition-* text-* hover:bg-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/walmart' not enclosed in quotes in tag <a class="block px-* py-* w-* text-* transition-* text-* hover:bg-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/proxy-mode/' not enclosed in quotes in tag <a class="block w-* py-* px-* text-* transition-* hover:bg-* text-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/chatgpt' not enclosed in quotes in tag <a class="block px-* py-* w-* text-* transition-* text-* hover:bg-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/google-api' not enclosed in quotes in tag <a class="block px-* py-* w-* text-* transition-* text-* hover:bg-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/chatgpt' not enclosed in quotes in tag <a class="block py-* text-* hover:underline">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/' not enclosed in quotes in tag <a class="max-* hidden items-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/amazon' not enclosed in quotes in tag <a class="block px-* py-* w-* text-* transition-* text-* hover:bg-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/amazon' not enclosed in quotes in tag <a class="block py-* text-* hover:underline">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/data-extraction/' not enclosed in quotes in tag <a class="block px-* py-* rounded transition-* xl:px-* hover:opacity-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '#' not enclosed in quotes in tag <a class="flex gap-* items-* px-* py-* rounded transition-* xl:px-* hover:opacity-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/images/favico.svg' not enclosed in quotes in tag <link ***>URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/js-scenario/' not enclosed in quotes in tag <a class="block px-* py-* rounded transition-* xl:px-* hover:opacity-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/fast-search' not enclosed in quotes in tag <a class="block py-* text-* hover:underline">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value 'https://dashboard.scrapingbee.com/account/register' not enclosed in quotes in tag <a class="block px-* py-* w-* text-* text-* text-* rounded transition-* bg-* hover:opacity-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/js-scenario/' not enclosed in quotes in tag <a class="block w-* py-* px-* text-* transition-* hover:bg-* text-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/google-api' not enclosed in quotes in tag <a class="block py-* text-* hover:underline">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'src' has a value 'https://cdn.jsdelivr.net/npm/alpinejs@3.x.x/dist/cdn.min.js' not enclosed in quotes in tag <script>URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/youtube' not enclosed in quotes in tag <a class="block px-* py-* w-* text-* transition-* text-* hover:bg-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/fast-search' not enclosed in quotes in tag <a class="block px-* py-* w-* text-* transition-* text-* hover:bg-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/images/favico.png' not enclosed in quotes in tag <link>URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/data-extraction/' not enclosed in quotes in tag <a class="block w-* py-* px-* text-* transition-* hover:bg-* text-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/' not enclosed in quotes in tag <a class="block px-* py-* rounded transition-* xl:px-* hover:opacity-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'src' has a value 'https://analytics.ahrefs.com/analytics.js' not enclosed in quotes in tag <script>URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/proxy-mode/' not enclosed in quotes in tag <a class="block px-* py-* rounded transition-* xl:px-* hover:opacity-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value 'https://help.scrapingbee.com/en/' not enclosed in quotes in tag <a class="block px-* py-* w-* text-* transition-* text-* hover:bg-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'href' has a value '/documentation/' not enclosed in quotes in tag <a class="block w-* py-* px-* text-* transition-* hover:bg-* text-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11The attribute 'src' has a value '/scripts/app-doc.min.js' not enclosed in quotes in tag <script>URL 1, URL 2, URL 3, URL 4, URL 5
warning7The attribute 'href' has a value 'https://dashboard.scrapingbee.com/account/manage/api_key' not enclosed in quotes in tag <a>URL 1, URL 2, URL 3, URL 4, URL 5
warning7The attribute 'href' has a value 'https://dashboard.scrapingbee.com/account/register' not enclosed in quotes in tag <a>URL 1, URL 2, URL 3, URL 4, URL 5
warning4The attribute 'href' has a value '#javascript-rendering' not enclosed in quotes in tag <a>URL 1, URL 2
warning4The attribute 'href' has a value '#search' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">URL 1, URL 2, URL 3, URL 4
warning4The attribute 'href' has a value '#country_code' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">URL 1, URL 2, URL 3, URL 4
warning4The attribute 'href' has a value '#language' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">URL 1, URL 2, URL 3, URL 4
warning3The attribute 'href' has a value '#light_request' not enclosed in quotes in tag <a>URL 1, URL 2, URL 3
warning3The attribute 'href' has a value '#api_key' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">URL 1, URL 2, URL 3
warning3The attribute 'href' has a value '/blog' not enclosed in quotes in tag <a>/documentation/data-extraction/
warning3The attribute 'href' has a value '#credit-cost-for-your-requests' not enclosed in quotes in tag <a>URL 1, URL 2
warning3The attribute 'href' has a value '#sort_by' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">URL 1, URL 2, URL 3
warning3The attribute 'href' has a value '#query' not enclosed in quotes in tag <a>URL 1, URL 2
warning3The attribute 'href' has a value '#ai_selector' not enclosed in quotes in tag <a>URL 1, URL 2
warning3The attribute 'href' has a value '/documentation/country_codes/' not enclosed in quotes in tag <a>URL 1, URL 2, URL 3
warning3The attribute 'href' has a value '#search' not enclosed in quotes in tag <a>URL 1, URL 2, URL 3
warning2The attribute 'href' has a value '#page' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">URL 1, URL 2
warning2The attribute 'href' has a value '/download/walmart_stores.json' not enclosed in quotes in tag <a>/documentation/walmart/
warning2The attribute 'href' has a value '#device' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">URL 1, URL 2
warning2The attribute 'href' has a value '#query' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">URL 1, URL 2
warning2The attribute 'href' has a value 'https://www.scrapingbee.com/' not enclosed in quotes in tag <a>URL 1, URL 2
warning2The attribute 'href' has a value '#js_scenario' not enclosed in quotes in tag <a>/documentation/
warning2The attribute 'href' has a value '#start_page' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">URL 1, URL 2
warning2The attribute 'href' has a value 'https://help.scrapingbee.com/en/article/url-encoding-1w5vas8/' not enclosed in quotes in tag <a>URL 1, URL 2
warning2The attribute 'href' has a value '/blog/' not enclosed in quotes in tag <a>URL 1, URL 2
warning2The attribute 'href' has a value '/documentation' not enclosed in quotes in tag <a>URL 1, URL 2
warning2The attribute 'href' has a value 'https://en.wikipedia.org/wiki/ISO_3166-***' not enclosed in quotes in tag <a>URL 1, URL 2
warning2The attribute 'href' has a value '#light_request' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">URL 1, URL 2
warning1The attribute 'href' has a value '#fulfillment_type' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/walmart/
warning1The attribute 'href' has a value '#hd' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/youtube/
warning1The attribute 'href' has a value '#subtitles' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/youtube/
warning1The attribute 'href' has a value '#js_scenario' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/
warning1The attribute 'href' has a value '#wait_for' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/
warning1The attribute 'href' has a value '/documentation/data-extraction/' not enclosed in quotes in tag <a>/documentation/
warning1The attribute 'href' has a value '#search_type' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/google-api/
warning1The attribute 'href' has a value '#purchased' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/youtube/
warning1The attribute 'href' has a value '#nfpr' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/google-api/
warning1The attribute 'href' has a value 'https://www.scrapingbee.com/documentation/google-api/' not enclosed in quotes in tag <link>/documentation/google-api/
warning1The attribute 'href' has a value 'https://dashboard.scrapingbee.com/dashboard' not enclosed in quotes in tag <a>/documentation/proxy-mode/
warning1The attribute 'href' has a value '#prompt' not enclosed in quotes in tag <a>/documentation/chatgpt/
warning1The attribute 'href' has a value '#3d' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/youtube/
warning1The attribute 'href' has a value '#ai_extract_rules' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/
warning1The attribute 'href' has a value '#premium_proxy' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/
warning1The attribute 'href' has a value '#screenshot_AmazonProduct' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/amazon/
warning1The attribute 'href' has a value 'https://www.scrapingbee.com/documentation/#stealth_proxy' not enclosed in quotes in tag <a>/documentation/js-scenario/
warning1The attribute 'href' has a value '#query_AmazonSearch' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/amazon/
warning1The attribute 'href' has a value '#domain_AmazonSearch' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/amazon/
warning1The attribute 'href' has a value '#vr***' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/youtube/
warning1The attribute 'href' has a value '#wait' not enclosed in quotes in tag <a>/documentation/
warning1The attribute 'href' has a value '#custom_google' not enclosed in quotes in tag <a>/documentation/
warning1The attribute 'href' has a value '#light_request_WalmartAPISearch' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/walmart/
warning1The attribute 'href' has a value '#ai_extract_rules' not enclosed in quotes in tag <a>/documentation/
warning1The attribute 'href' has a value '#category_id' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/amazon/
warning1The attribute 'href' has a value '#render_js' not enclosed in quotes in tag <a>/documentation/
warning1The attribute 'href' has a value '#own_proxy' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/
warning1The attribute 'href' has a value '#session_id' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/
warning1The attribute 'href' has a value '#add_html_WalmartAPISearch' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/walmart/
warning1The attribute 'href' has a value '#ai_query' not enclosed in quotes in tag <a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*">/documentation/
warning1The attribute 'src' has a value '/images/documentation/proxy-mode/apify_2.png' not enclosed in quotes in tag <img ***>/documentation/proxy-mode/
You have reached the limit of 100 rows as a protection against very large output or exhausted memory.
No rows found, please edit your search term.

DOM depth

No problems found.


Heading structure

No problems found.


Non-clickable phone numbers

SeverityOccursDetailAffected URLs (max 5)
warning1(212) 831-0300/documentation/google-api/

Title uniqueness

No problems found.


Description uniqueness

No problems found.

Accessibility

Analysis nameOKNoticeWarningCritical
Missing aria labels1082
Missing roles0060
Missing form labels0010
Missing image alt attributes8020
Missing html lang attribute1000

Valid HTML

No problems found.


Missing image alt attributes

SeverityOccursDetailAffected URLs (max 5)
warning260<img class="flex-* w-* h-*">URL 1, URL 2, URL 3, URL 4, URL 5

Missing form labels

SeverityOccursDetailAffected URLs (max 5)
warning11<input ***>URL 1, URL 2, URL 3, URL 4, URL 5

Missing aria labels

Found 20 row(s).
SeverityOccursDetailAffected URLs (max 5)
critical11<input ***>URL 1, URL 2, URL 3, URL 4, URL 5
critical2<select class="style-*">/documentation/
warning130<button class="{ 'bg-* selectedLibrary === library.value }" class="flex gap-* items-* px-* py-* w-* transition-* hover:bg-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning130<button class="flex justify-* items-* px-* py-* bg-* rounded-* border border-* transition-* hover:bg-* focus:outline-* min-* shadow-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning128<a class="bg-* border border-* py-* px-* rounded-* !no-* text-* inline-* items-* gap-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning88<a class="block py-* text-* hover:underline" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning88<a class="block px-* py-* w-* text-* transition-* text-* hover:bg-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning78<a ***>URL 1, URL 2, URL 3, URL 4, URL 5
warning44<a class="block px-* py-* rounded transition-* xl:px-* hover:opacity-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning44<a class="block w-* py-* px-* text-* transition-* hover:bg-* text-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning22<button class="flex justify-* items-* px-* py-* w-* text-* transition-* text-* hover:bg-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning11<a class="px-* py-* shrink-* !font-* text-* rounded transition-* bg-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning11<a class="block px-* py-* w-* text-* text-* text-* rounded transition-* bg-* hover:opacity-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning11<a class="max-* hidden items-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning11<a class="flex gap-* items-* px-* py-* rounded transition-* xl:px-* hover:opacity-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning11<a class="flex gap-* items-* px-* py-* rounded transition-* lg:px-* hover:opacity-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning11<button class="flex items-* justify-* gap-* px-* py-* rounded border text-* text-* text-* transition-* hover:opacity-* shadow-*" id="custom-search-trigger" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning11<a class="px-* py-* rounded transition-* shrink-* xl:px-* hover:opacity-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning11<a class="hidden items-* lg:flex" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning11<a class="block px-* py-* w-* text-* text-* transition-* text-* hover:bg-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
No rows found, please edit your search term.

Missing roles

SeverityOccursDetailAffected URLs (max 5)
warning11<aside class="hidden w-* p-* md:flex flex-* h-* gap-* border-* border-* max-* lg:max-* overflow-* 2xl:max-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11<header class="border-* p-* border-* bg-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11<nav>URL 1, URL 2, URL 3, URL 4, URL 5
warning11<main class="overflow-* flex-* bg-*">URL 1, URL 2, URL 3, URL 4, URL 5
warning11<nav class="flex xl:!hidden justify-* items-* w-*" *** >URL 1, URL 2, URL 3, URL 4, URL 5
warning11<nav class="hidden xl:flex items-* gap-* text-* text-*">URL 1, URL 2, URL 3, URL 4, URL 5

Missing html lang attribute

No problems found.

Security

Found 10 row(s).
HeaderOKNoticeWarningCriticalRecommendation
Strict-Transport-Security00011Strict-Transport-Security header is not set. It enforces secure connections and protects against MITM attacks.
Content-Security-Policy00011Content-Security-Policy header is not set. It restricts resources the page can load and prevents XSS attacks.
Access-Control-Allow-Origin00110Access-Control-Allow-Origin is set to '*' which allows any origin to access the resource. This can be a security risk.
X-Frame-Options00110X-Frame-Options header is not set. It prevents clickjacking attacks when set to 'deny' or 'sameorigin.
Feature-Policy00110Feature-Policy header is not set. It allows enabling/disabling browser APIs and features for security. Not important if Permissions-Policy is set.
Permissions-Policy00110Permissions-Policy header is not set. It allows enabling/disabling browser APIs and features for security.
Server01100Server header is set to 'cloudflare'. It is better not to reveal used technologies.
X-XSS-Protection11000
X-Content-Type-Options11000
Referrer-Policy11000
No rows found, please edit your search term.

Security headers

SeverityOccursDetailAffected URLs (max 5)
critical11Strict-Transport-Security header is not set. It enforces secure connections and protects against MITM attacks.URL 1, URL 2, URL 3, URL 4, URL 5
critical11Content-Security-Policy header is not set. It restricts resources the page can load and prevents XSS attacks.URL 1, URL 2, URL 3, URL 4, URL 5
warning11X-Frame-Options header is not set. It prevents clickjacking attacks when set to 'deny' or 'sameorigin.URL 1, URL 2, URL 3, URL 4, URL 5
warning11Access-Control-Allow-Origin is set to '*' which allows any origin to access the resource. This can be a security risk.URL 1, URL 2, URL 3, URL 4, URL 5
warning11Feature-Policy header is not set. It allows enabling/disabling browser APIs and features for security. Not important if Permissions-Policy is set.URL 1, URL 2, URL 3, URL 4, URL 5
warning11Permissions-Policy header is not set. It allows enabling/disabling browser APIs and features for security.URL 1, URL 2, URL 3, URL 4, URL 5
notice11Server header is set to 'cloudflare'. It is better not to reveal used technologies.URL 1, URL 2, URL 3, URL 4, URL 5

TOP non-unique titles

Nothing to report.


TOP non-unique descriptions

Count 🔽Description
11

SEO metadata

Found 11 row(s).
URL 🔼IndexingTitleH1DescriptionKeywords
/documentation/AllowedHTML APIHTML APIScrapingBee is the best web scraping API that handles proxies and headless browsers for you — so you can focus on extracting the data you need.
/documentation/amazon/AllowedAmazon APIAmazon APIOur Amazon API allows you to scrape Amazon search results and product details in realtime. We provide two endpoints: Search endpoint (/api/v1/amazon/search) - Fetch Amazon search results Product endpoint (/api/v1/amazon/product) - Fetch structured Amazon product details Amazon Product API Quick start To scrape Amazon product details, you only need two things: your API key, available here a product ASIN (learn more about ASIN) Then, simply do this. Copy curl "https://app.scrapingbee.com/api/v1/amazon/product?api_key=YOUR-API-KEY&query=B0DPDRNSXV" # Install the Python Requests library: # pip install requests import requests def send_request(): response = requests.get( url='https://app.scrapingbee.com/api/v1/amazon/product', params={ 'api_key': 'YOUR-API-KEY', 'query': 'B0DPDRNSXV', }, ) print('Response HTTP Status Code: ', response.status_code) print('Response HTTP Response Body: ', response.content) send_request() // Install the Node Axios package // npm install axios const axios = require('axios'); axios.get('https://app.scrapingbee.com/api/v1/amazon/product', { params: { 'api_key': 'YOUR-API-KEY', 'url': 'YOUR-URL', 'query': B0DPDRNSXV, } }).then(function (response) { // handle success console.log(response); }) import java.io.IOException; import org.apache.http.client.fluent.*; public class SendRequest { public static void main(String[] args) { sendRequest(); } private static void sendRequest() { // Classic (GET ) try { // Create request Content content = Request.Get("https://app.scrapingbee.com/api/v1/amazon/product?api_key=YOUR-API-KEY&url=YOUR-URL&query=B0DPDRNSXV") // Fetch request and return content .execute().returnContent(); // Print content System.out.println(content); } catch (IOException e) { System.out.println(e); } } } require 'net/http' require 'net/https' # Classic (GET ) def send_request uri = URI('https://app.scrapingbee.com/api/v1/amazon/product?api_key=YOUR-API-KEY&url=YOUR-URL&query=B0DPDRNSXV') # Create client http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_PEER # Create Request req = Net::HTTP::Get.new(uri) # Fetch Request res = http.request(req) puts "Response HTTP Status Code: #{ res.code }" puts "Response HTTP Response Body: #{ res.body }" rescue StandardError => e puts "HTTP Request failed (#{ e.message })" end send_request() <?php // get cURL resource $ch = curl_init(); // set url curl_setopt($ch, CURLOPT_URL, 'https://app.scrapingbee.com/api/v1/amazon/product?api_key=YOUR-API-KEY&url=YOUR-URL&query=B0DPDRNSXV'); // set method curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET'); // return the transfer as a string curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // send the request and save response to $response $response = curl_exec($ch); // stop if fails if (!$response) { die('Error: "' . curl_error($ch) . '" - Code: ' . curl_errno($ch)); } echo 'HTTP Status Code: ' . curl_getinfo($ch, CURLINFO_HTTP_CODE) . PHP_EOL; echo 'Response Body: ' . $response . PHP_EOL; // close curl resource to free up system resources curl_close($ch); > package main import ( "fmt" "io/ioutil" "net/http" ) func sendClassic() { // Create client client := &http.Client{} // Create request req, err := http.NewRequest("GET", "https://app.scrapingbee.com/api/v1/amazon/product?api_key=YOUR-API-KEY&url=YOUR-URL&query=B0DPDRNSXV", nil) parseFormErr := req.ParseForm() if parseFormErr != nil { fmt.Println(parseFormErr) } // Fetch Request resp, err := client.Do(req) if err != nil { fmt.Println("Failure : ", err) } // Read Response Body respBody, _ := ioutil.ReadAll(resp.Body) // Display Results fmt.Println("response Status : ", resp.Status) fmt.Println("response Headers : ", resp.Header) fmt.Println("response Body : ", string(respBody)) } func main() { sendClassic() } Here is a breakdown of all the parameters you can use with the Amazon Product API:
/documentation/chatgpt/AllowedGPT APIGPT APIOur Chat GPT API allows you to send prompts to a GPT model and receive AI-generated responses in realtime. We provide one endpoint: GPT endpoint (/api/v1/chatgpt) - Send prompts to GPT and receive AI-generated responses Quick start To use the GPT API, you only need two things: your API key, available here a prompt to send to the GPT model (learn more about prompts) Then, simply do this. Copy curl "https://app.scrapingbee.com/api/v1/chatgpt?api_key=YOUR-API-KEY&prompt=Explain+the+benefits+of+renewable+energy+in+100+words" # Install the Python Requests library: # pip install requests import requests def send_request(): response = requests.get( url='https://app.scrapingbee.com/api/v1/chatgpt', params={ 'api_key': 'YOUR-API-KEY', 'prompt': 'Explain the benefits of renewable energy in 100 words', }, ) print('Response HTTP Status Code: ', response.status_code) print('Response HTTP Response Body: ', response.content) send_request() // Install the Node Axios package // npm install axios const axios = require('axios'); axios.get('https://app.scrapingbee.com/api/v1/chatgpt', { params: { 'api_key': 'YOUR-API-KEY', 'url': 'YOUR-URL', 'prompt': Explain the benefits of renewable energy in 100 words, } }).then(function (response) { // handle success console.log(response); }) import java.io.IOException; import org.apache.http.client.fluent.*; public class SendRequest { public static void main(String[] args) { sendRequest(); } private static void sendRequest() { // Classic (GET ) try { // Create request Content content = Request.Get("https://app.scrapingbee.com/api/v1/chatgpt?api_key=YOUR-API-KEY&url=YOUR-URL&prompt=Explain+the+benefits+of+renewable+energy+in+100+words") // Fetch request and return content .execute().returnContent(); // Print content System.out.println(content); } catch (IOException e) { System.out.println(e); } } } require 'net/http' require 'net/https' # Classic (GET ) def send_request uri = URI('https://app.scrapingbee.com/api/v1/chatgpt?api_key=YOUR-API-KEY&url=YOUR-URL&prompt=Explain+the+benefits+of+renewable+energy+in+100+words') # Create client http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_PEER # Create Request req = Net::HTTP::Get.new(uri) # Fetch Request res = http.request(req) puts "Response HTTP Status Code: #{ res.code }" puts "Response HTTP Response Body: #{ res.body }" rescue StandardError => e puts "HTTP Request failed (#{ e.message })" end send_request() <?php // get cURL resource $ch = curl_init(); // set url curl_setopt($ch, CURLOPT_URL, 'https://app.scrapingbee.com/api/v1/chatgpt?api_key=YOUR-API-KEY&url=YOUR-URL&prompt=Explain+the+benefits+of+renewable+energy+in+100+words'); // set method curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET'); // return the transfer as a string curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // send the request and save response to $response $response = curl_exec($ch); // stop if fails if (!$response) { die('Error: "' . curl_error($ch) . '" - Code: ' . curl_errno($ch)); } echo 'HTTP Status Code: ' . curl_getinfo($ch, CURLINFO_HTTP_CODE) . PHP_EOL; echo 'Response Body: ' . $response . PHP_EOL; // close curl resource to free up system resources curl_close($ch); > package main import ( "fmt" "io/ioutil" "net/http" ) func sendClassic() { // Create client client := &http.Client{} // Create request req, err := http.NewRequest("GET", "https://app.scrapingbee.com/api/v1/chatgpt?api_key=YOUR-API-KEY&url=YOUR-URL&prompt=Explain+the+benefits+of+renewable+energy+in+100+words", nil) parseFormErr := req.ParseForm() if parseFormErr != nil { fmt.Println(parseFormErr) } // Fetch Request resp, err := client.Do(req) if err != nil { fmt.Println("Failure : ", err) } // Read Response Body respBody, _ := ioutil.ReadAll(resp.Body) // Display Results fmt.Println("response Status : ", resp.Status) fmt.Println("response Headers : ", resp.Header) fmt.Println("response Body : ", string(respBody)) } func main() { sendClassic() } Here is a breakdown of all the parameters you can use with the GPT API:
/documentation/country_codes/AllowedSupported CountriesSupported CountriesList of supported country codes The following is the list of supported country codes using ISO 3166-1 format . Use country code with the country_code parameter. Geolocation is only available when premium proxies are enabled: premium_proxy=true. Country Namecountry_codeAfghanistanafAlbaniaalAlgeriadzAmerican SamoaasAndorraadAngolaaoAnguillaaiAntarcticaaqAntigua & BarbudaagArgentinaarArmeniaamArubaawAustraliaauAustriaatAzerbaijanazBahamabsBahrainbhBangladeshbdBarbadosbbBelarusbyBelgiumbeBelizebzBeninbjBermudabmBhutanbtBoliviaboBosnia and HerzegovinabaBotswanabwBouvet IslandbvBrazilbrBritish Indian Ocean TerritoryioBritish Virgin IslandsvgBrunei DarussalambnBulgariabgBurkina FasobfBurma (no longer exists)buBurundibiCambodiakhCamerooncmCanadacaCape VerdecvCayman IslandskyCentral African RepubliccfChadtdChileclChinacnChristmas IslandcxCocos (Keeling) IslandsccColombiacoComoroskmCongocgCook IislandsckCosta RicacrCroatiahrCubacuCypruscyCzech RepublicczCzechoslovakia (no longer exists)csCôte D'ivoire (Ivory Coast)ciDemocratic Yemen (no longer exists)ydDenmarkdkDjiboutidjDominicadmDominican RepublicdoEast TimortpEcuadorecEgyptegEl SalvadorsvEquatorial GuineagqEritreaerEstoniaeeEthiopiaetFalkland Islands (Malvinas)fkFaroe IslandsfoFijifjFinlandfiFrancefrFrench GuianagfFrench PolynesiapfFrench Southern TerritoriestfGabongaGambiagmGeorgiageGerman Democratic Republic (no longer exists)ddGermanydeGhanaghGibraltargiGreecegrGreenlandglGrenadagdGuadeloupegpGuamguGuatemalagtGuineagnGuinea-BissaugwGuyanagyHaitihtHeard & McDonald IslandshmHondurashnHong KonghkHungaryhuIcelandisIndiainIndonesiaidIraqiqIrelandieIslamic Republic of IranirIsraelilItalyitJamaicajmJapanjpJordanjoKazakhstankzKenyakeKiribatikiKorea, Democratic People's Republic ofkpKorea, Republic ofkrKuwaitkwKyrgyzstankgLao People's Democratic RepubliclaLatvialvLebanonlbLesotholsLiberialrLibyan Arab JamahiriyalyLiechtensteinliLithuanialtLuxembourgluMacaumoMadagascarmgMalawimwMalaysiamyMaldivesmvMalimlMaltamtMarshall IslandsmhMartiniquemqMauritaniamrMauritiusmuMayotteytMexicomxMicronesiafmMoldova, Republic ofmdMonacomcMongoliamnMonserratmsMoroccomaMozambiquemzMyanmarmmNamibianaNaurunrNepalnpNetherlands AntillesanNetherlandsnlNeutral Zone (no longer exists)ntNew CaledoniancNew ZealandnzNicaraguaniNigerneNigeriangNiuenuNorfolk IslandnfNorthern Mariana IslandsmpNorwaynoOmanomPakistanpkPalaupwPanamapaPapua New GuineapgParaguaypyPerupePhilippinesphPitcairnpnPolandplPortugalptPuerto RicoprQatarqaRomaniaroRussian FederationruRwandarwRéunionreSaint LucialcSamoawsSan MarinosmSao Tome & PrincipestSaudi ArabiasaSenegalsnSeychellesscSierra LeoneslSingaporesgSlovakiaskSloveniasiSolomon IslandssbSomaliasoSouth AfricazaSouth Georgia and the South Sandwich IslandsgsSpainesSri LankalkSt. HelenashSt. Kitts and NevisknSt. Pierre & MiquelonpmSt. Vincent & the GrenadinesvcSudansdSurinamesrSvalbard & Jan Mayen IslandssjSwazilandszSwedenseSwitzerlandchSyrian Arab RepublicsyTaiwan, Province of ChinatwTajikistantjTanzania, United Republic oftzThailandthTogotgTokelautkTongatoTrinidad & TobagottTunisiatnTurkeytrTurkmenistantmTurks & Caicos IslandstcTuvalutvUgandaugUkraineuaUnion of Soviet Socialist Republics (no longer exists)suUnited Arab EmiratesaeUnited Kingdom (Great Britain)gbUnited States Minor Outlying IslandsumUnited States Virgin IslandsviUnited StatesusUruguayuyUzbekistanuzVanuatuvuVatican City State (Holy See)vaVenezuelaveViet NamvnWallis & Futuna IslandswfWestern SaharaehYemenyeYugoslaviayuZairezrZambiazmZimbabwezw
/documentation/data-extraction/AllowedData ExtractionData Extraction 💡 Important:This page explains how to use a specific feature of our main web scraping API!If you are not yet familiar with ScrapingBee web scraping API, you can read the documentation here. Basic usage If you want to extract data from pages and don't want to parse the HTML on your side, you can add extraction rules to your API call. The simplest way to use extraction rules is to use the following format
/documentation/fast-search/AllowedFast Search APIFast Search API name [type] (default) Description api_key [string] required Your api key Learn more search [string] required The text you would put in the search bar Learn more country_code [string] ("us") Country code used to localize results (ISO 3166-1 alpha-2) Learn more language [string] ("en") Language of the search results (e.g. en, fr) Learn more page [integer] (1) The page number you want to extract results from Learn more Getting Started Our Fast Search API delivers search results in under a second, with a simple, lightweight request.
/documentation/google-api/AllowedGoogle APIGoogle API name [type] (default) Description api_key [string] required Your api key Learn more search [string] required The text you would put in the Google search bar Learn more add_html [boolean] (false) Adding the full html of the page in the results Learn more country_code [string] ("us") Country code from which you would like the request to come from Learn more device ["desktop" | "mobile"] ("desktop") Control the device the request will be sent from Learn more extra_params [string] ("") Extra Google URL parameters Learn more language [string] ("en") Language the search results will be displayed in Learn more light_request [boolean] (true) Light requests are faster and cheaper (10 credits instead of 15), but some content may be missing. Learn more nfpr [boolean] (false) Exclude results from auto-corrected queries that were spelt wrong. Learn more page [integer] (1) The page number you want to extract results from Learn more search_type ["classic" | "news" | "maps" | "images" | "lens" | "shopping" | "ai_mode"] ("classic") The type of search you want to perform Learn more Getting Started Our Google Search API allows you to scrape search results pages in realtime.
/documentation/js-scenario/AllowedJavaScript ScenarioJavaScript Scenario 💡 Important:This page explains how to use a specific feature of our main web scraping API!If you are not yet familiar with ScrapingBee web scraping API, you can read the documentation here. Basic usage If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call. For example, if you wish to click on a button, you will need to use this scenario.
/documentation/proxy-mode/AllowedProxy ModeProxy ModeWhat is the proxy mode? ScrapingBee also offers a proxy front-end to the API. This can make integration with third-party tools easier. The Proxy mode only changes the way you access ScrapingBee. The ScrapingBee API will then handle requests just like any standard request. Request cost, return code and default parameters will be the same as a standard no-proxy request. We recommend disabling Javascript rendering in proxy mode, which is enabled by default. The following credentials and configurations are used to access the proxy mode:
/documentation/walmart/AllowedWalmart APIWalmart APIOur Walmart API allows you to scrape Walmart search results and product details in realtime. We provide two endpoints: Search endpoint (/api/v1/walmart/search) - Fetch Walmart search results Product endpoint (/api/v1/walmart/product) - Fetch structured Walmart product details Walmart Search API Quick start To scrape Walmart search results, you only need two things: your API key, available here a search query (learn more about search query) Then, simply do this. Copy curl "https://app.scrapingbee.com/api/v1/walmart/search?api_key=YOUR-API-KEY&query=iphone" # Install the Python Requests library: # `pip install requests` import requests def send_request(): response = requests.get( url="https://app.scrapingbee.com/api/v1/walmart/search", params={ "api_key": "YOUR-API-KEY", "query": "iphone", }, ) print('Response HTTP Status Code: ', response.status_code) print('Response HTTP Response Body: ', response.content) send_request() // request Axios const axios = require('axios'); axios.get('https://app.scrapingbee.com/api/v1/walmart/search', { params: { 'api_key': 'YOUR-API-KEY', 'query': 'iphone', } }).then(function (response) { // handle success console.log(response); }) import java.io.IOException; import org.apache.http.client.fluent.*; public class SendRequest { public static void main(String[] args) { sendRequest(); } private static void sendRequest() { // Classic (GET ) try { // Create request Content content = Request.Get("https://app.scrapingbee.com/api/v1/walmart/search?api_key=YOUR-API-KEY&query=iphone") // Fetch request and return content .execute().returnContent(); // Print content System.out.println(content); } catch (IOException e) { System.out.println(e); } } } require 'net/http' require 'net/https' # Classic (GET ) def send_request uri = URI('https://app.scrapingbee.com/api/v1/walmart/search?api_key=YOUR-API-KEY&query=iphone') # Create client http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_PEER # Create Request req = Net::HTTP::Get.new(uri) # Fetch Request res = http.request(req) puts "Response HTTP Status Code: #{ res.code }" puts "Response HTTP Response Body: #{ res.body }" rescue StandardError => e puts "HTTP Request failed (#{ e.message })" end send_request() <?php // get cURL resource $ch = curl_init(); // set url curl_setopt($ch, CURLOPT_URL, 'https://app.scrapingbee.com/api/v1/walmart/search?api_key=YOUR-API-KEY&query=iphone'); // set method curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET'); // return the transfer as a string curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // send the request and save response to $response $response = curl_exec($ch); // stop if fails if (!$response) { die('Error: "' . curl_error($ch) . '" - Code: ' . curl_errno($ch)); } echo 'HTTP Status Code: ' . curl_getinfo($ch, CURLINFO_HTTP_CODE) . PHP_EOL; echo 'Response Body: ' . $response . PHP_EOL; // close curl resource to free up system resources curl_close($ch); ?> package main import ( "fmt" "io/ioutil" "net/http" ) func sendClassic() { // Create client client := &http.Client{} // Create request req, err := http.NewRequest("GET", "https://app.scrapingbee.com/api/v1/walmart/search?api_key=YOUR-API-KEY&query=iphone", nil) parseFormErr := req.ParseForm() if parseFormErr != nil { fmt.Println(parseFormErr) } // Fetch Request resp, err := client.Do(req) if err != nil { fmt.Println("Failure : ", err) } // Read Response Body respBody, _ := ioutil.ReadAll(resp.Body) // Display Results fmt.Println("response Status : ", resp.Status) fmt.Println("response Headers : ", resp.Header) fmt.Println("response Body : ", string(respBody)) } func main() { sendClassic() } Here is a breakdown of all the parameters you can use with the Walmart Search API:
/documentation/youtube/AllowedYouTube APIYouTube APIOur YouTube API allows you to scrape YouTube search results, video metadata, transcripts, and trainability information in realtime. We provide four endpoints: Search endpoint (/api/v1/youtube/search) - Fetch YouTube search results Metadata endpoint (/api/v1/youtube/metadata) - Fetch structured YouTube video metadata Transcript endpoint (/api/v1/youtube/transcript) - Fetch YouTube video transcripts Trainability endpoint (/api/v1/youtube/trainability) - Check video transcript availability YouTube Search API Quick start To scrape YouTube search results, you only need two things:
No rows found, please edit your search term.

OpenGraph metadata

Found 11 row(s).
URL 🔼OG TitleOG DescriptionOG Image
/documentation/HTML API | ScrapingBee ScrapingBee is the best web scraping API that handles proxies and headless browsers for you — so you can focus on extracting the data you need./images/cover.png
/documentation/amazon/Amazon API | ScrapingBee Our Amazon API allows you to scrape Amazon search results and product details in realtime. We provide two endpoints: Search endpoint (/api/v1/amazon/search) - Fetch Amazon search results Product endpoint (/api/v1/amazon/product) - Fetch structured Amazon product details Amazon Product API Quick start To scrape Amazon product details, you only need two things: your API key, available here a product ASIN (learn more about ASIN) Then, simply do this. Copy curl "https://app.scrapingbee.com/api/v1/amazon/product?api_key=YOUR-API-KEY&query=B0DPDRNSXV" # Install the Python Requests library: # pip install requests import requests def send_request(): response = requests.get( url='https://app.scrapingbee.com/api/v1/amazon/product', params={ 'api_key': 'YOUR-API-KEY', 'query': 'B0DPDRNSXV', }, ) print('Response HTTP Status Code: ', response.status_code) print('Response HTTP Response Body: ', response.content) send_request() // Install the Node Axios package // npm install axios const axios = require('axios'); axios.get('https://app.scrapingbee.com/api/v1/amazon/product', { params: { 'api_key': 'YOUR-API-KEY', 'url': 'YOUR-URL', 'query': B0DPDRNSXV, } }).then(function (response) { // handle success console.log(response); }) import java.io.IOException; import org.apache.http.client.fluent.*; public class SendRequest { public static void main(String[] args) { sendRequest(); } private static void sendRequest() { // Classic (GET ) try { // Create request Content content = Request.Get("https://app.scrapingbee.com/api/v1/amazon/product?api_key=YOUR-API-KEY&url=YOUR-URL&query=B0DPDRNSXV") // Fetch request and return content .execute().returnContent(); // Print content System.out.println(content); } catch (IOException e) { System.out.println(e); } } } require 'net/http' require 'net/https' # Classic (GET ) def send_request uri = URI('https://app.scrapingbee.com/api/v1/amazon/product?api_key=YOUR-API-KEY&url=YOUR-URL&query=B0DPDRNSXV') # Create client http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_PEER # Create Request req = Net::HTTP::Get.new(uri) # Fetch Request res = http.request(req) puts "Response HTTP Status Code: #{ res.code }" puts "Response HTTP Response Body: #{ res.body }" rescue StandardError => e puts "HTTP Request failed (#{ e.message })" end send_request() <?php // get cURL resource $ch = curl_init(); // set url curl_setopt($ch, CURLOPT_URL, 'https://app.scrapingbee.com/api/v1/amazon/product?api_key=YOUR-API-KEY&url=YOUR-URL&query=B0DPDRNSXV'); // set method curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET'); // return the transfer as a string curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // send the request and save response to $response $response = curl_exec($ch); // stop if fails if (!$response) { die('Error: "' . curl_error($ch) . '" - Code: ' . curl_errno($ch)); } echo 'HTTP Status Code: ' . curl_getinfo($ch, CURLINFO_HTTP_CODE) . PHP_EOL; echo 'Response Body: ' . $response . PHP_EOL; // close curl resource to free up system resources curl_close($ch); > package main import ( "fmt" "io/ioutil" "net/http" ) func sendClassic() { // Create client client := &http.Client{} // Create request req, err := http.NewRequest("GET", "https://app.scrapingbee.com/api/v1/amazon/product?api_key=YOUR-API-KEY&url=YOUR-URL&query=B0DPDRNSXV", nil) parseFormErr := req.ParseForm() if parseFormErr != nil { fmt.Println(parseFormErr) } // Fetch Request resp, err := client.Do(req) if err != nil { fmt.Println("Failure : ", err) } // Read Response Body respBody, _ := ioutil.ReadAll(resp.Body) // Display Results fmt.Println("response Status : ", resp.Status) fmt.Println("response Headers : ", resp.Header) fmt.Println("response Body : ", string(respBody)) } func main() { sendClassic() } Here is a breakdown of all the parameters you can use with the Amazon Product API:/images/cover.png
/documentation/chatgpt/GPT API | ScrapingBee Our Chat GPT API allows you to send prompts to a GPT model and receive AI-generated responses in realtime. We provide one endpoint: GPT endpoint (/api/v1/chatgpt) - Send prompts to GPT and receive AI-generated responses Quick start To use the GPT API, you only need two things: your API key, available here a prompt to send to the GPT model (learn more about prompts) Then, simply do this. Copy curl "https://app.scrapingbee.com/api/v1/chatgpt?api_key=YOUR-API-KEY&prompt=Explain+the+benefits+of+renewable+energy+in+100+words" # Install the Python Requests library: # pip install requests import requests def send_request(): response = requests.get( url='https://app.scrapingbee.com/api/v1/chatgpt', params={ 'api_key': 'YOUR-API-KEY', 'prompt': 'Explain the benefits of renewable energy in 100 words', }, ) print('Response HTTP Status Code: ', response.status_code) print('Response HTTP Response Body: ', response.content) send_request() // Install the Node Axios package // npm install axios const axios = require('axios'); axios.get('https://app.scrapingbee.com/api/v1/chatgpt', { params: { 'api_key': 'YOUR-API-KEY', 'url': 'YOUR-URL', 'prompt': Explain the benefits of renewable energy in 100 words, } }).then(function (response) { // handle success console.log(response); }) import java.io.IOException; import org.apache.http.client.fluent.*; public class SendRequest { public static void main(String[] args) { sendRequest(); } private static void sendRequest() { // Classic (GET ) try { // Create request Content content = Request.Get("https://app.scrapingbee.com/api/v1/chatgpt?api_key=YOUR-API-KEY&url=YOUR-URL&prompt=Explain+the+benefits+of+renewable+energy+in+100+words") // Fetch request and return content .execute().returnContent(); // Print content System.out.println(content); } catch (IOException e) { System.out.println(e); } } } require 'net/http' require 'net/https' # Classic (GET ) def send_request uri = URI('https://app.scrapingbee.com/api/v1/chatgpt?api_key=YOUR-API-KEY&url=YOUR-URL&prompt=Explain+the+benefits+of+renewable+energy+in+100+words') # Create client http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_PEER # Create Request req = Net::HTTP::Get.new(uri) # Fetch Request res = http.request(req) puts "Response HTTP Status Code: #{ res.code }" puts "Response HTTP Response Body: #{ res.body }" rescue StandardError => e puts "HTTP Request failed (#{ e.message })" end send_request() <?php // get cURL resource $ch = curl_init(); // set url curl_setopt($ch, CURLOPT_URL, 'https://app.scrapingbee.com/api/v1/chatgpt?api_key=YOUR-API-KEY&url=YOUR-URL&prompt=Explain+the+benefits+of+renewable+energy+in+100+words'); // set method curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET'); // return the transfer as a string curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // send the request and save response to $response $response = curl_exec($ch); // stop if fails if (!$response) { die('Error: "' . curl_error($ch) . '" - Code: ' . curl_errno($ch)); } echo 'HTTP Status Code: ' . curl_getinfo($ch, CURLINFO_HTTP_CODE) . PHP_EOL; echo 'Response Body: ' . $response . PHP_EOL; // close curl resource to free up system resources curl_close($ch); > package main import ( "fmt" "io/ioutil" "net/http" ) func sendClassic() { // Create client client := &http.Client{} // Create request req, err := http.NewRequest("GET", "https://app.scrapingbee.com/api/v1/chatgpt?api_key=YOUR-API-KEY&url=YOUR-URL&prompt=Explain+the+benefits+of+renewable+energy+in+100+words", nil) parseFormErr := req.ParseForm() if parseFormErr != nil { fmt.Println(parseFormErr) } // Fetch Request resp, err := client.Do(req) if err != nil { fmt.Println("Failure : ", err) } // Read Response Body respBody, _ := ioutil.ReadAll(resp.Body) // Display Results fmt.Println("response Status : ", resp.Status) fmt.Println("response Headers : ", resp.Header) fmt.Println("response Body : ", string(respBody)) } func main() { sendClassic() } Here is a breakdown of all the parameters you can use with the GPT API:/images/cover.png
/documentation/country_codes/Supported Countries | ScrapingBee List of supported country codes The following is the list of supported country codes using ISO 3166-1 format . Use country code with the country_code parameter. Geolocation is only available when premium proxies are enabled: premium_proxy=true. Country Namecountry_codeAfghanistanafAlbaniaalAlgeriadzAmerican SamoaasAndorraadAngolaaoAnguillaaiAntarcticaaqAntigua & BarbudaagArgentinaarArmeniaamArubaawAustraliaauAustriaatAzerbaijanazBahamabsBahrainbhBangladeshbdBarbadosbbBelarusbyBelgiumbeBelizebzBeninbjBermudabmBhutanbtBoliviaboBosnia and HerzegovinabaBotswanabwBouvet IslandbvBrazilbrBritish Indian Ocean TerritoryioBritish Virgin IslandsvgBrunei DarussalambnBulgariabgBurkina FasobfBurma (no longer exists)buBurundibiCambodiakhCamerooncmCanadacaCape VerdecvCayman IslandskyCentral African RepubliccfChadtdChileclChinacnChristmas IslandcxCocos (Keeling) IslandsccColombiacoComoroskmCongocgCook IislandsckCosta RicacrCroatiahrCubacuCypruscyCzech RepublicczCzechoslovakia (no longer exists)csCôte D'ivoire (Ivory Coast)ciDemocratic Yemen (no longer exists)ydDenmarkdkDjiboutidjDominicadmDominican RepublicdoEast TimortpEcuadorecEgyptegEl SalvadorsvEquatorial GuineagqEritreaerEstoniaeeEthiopiaetFalkland Islands (Malvinas)fkFaroe IslandsfoFijifjFinlandfiFrancefrFrench GuianagfFrench PolynesiapfFrench Southern TerritoriestfGabongaGambiagmGeorgiageGerman Democratic Republic (no longer exists)ddGermanydeGhanaghGibraltargiGreecegrGreenlandglGrenadagdGuadeloupegpGuamguGuatemalagtGuineagnGuinea-BissaugwGuyanagyHaitihtHeard & McDonald IslandshmHondurashnHong KonghkHungaryhuIcelandisIndiainIndonesiaidIraqiqIrelandieIslamic Republic of IranirIsraelilItalyitJamaicajmJapanjpJordanjoKazakhstankzKenyakeKiribatikiKorea, Democratic People's Republic ofkpKorea, Republic ofkrKuwaitkwKyrgyzstankgLao People's Democratic RepubliclaLatvialvLebanonlbLesotholsLiberialrLibyan Arab JamahiriyalyLiechtensteinliLithuanialtLuxembourgluMacaumoMadagascarmgMalawimwMalaysiamyMaldivesmvMalimlMaltamtMarshall IslandsmhMartiniquemqMauritaniamrMauritiusmuMayotteytMexicomxMicronesiafmMoldova, Republic ofmdMonacomcMongoliamnMonserratmsMoroccomaMozambiquemzMyanmarmmNamibianaNaurunrNepalnpNetherlands AntillesanNetherlandsnlNeutral Zone (no longer exists)ntNew CaledoniancNew ZealandnzNicaraguaniNigerneNigeriangNiuenuNorfolk IslandnfNorthern Mariana IslandsmpNorwaynoOmanomPakistanpkPalaupwPanamapaPapua New GuineapgParaguaypyPerupePhilippinesphPitcairnpnPolandplPortugalptPuerto RicoprQatarqaRomaniaroRussian FederationruRwandarwRéunionreSaint LucialcSamoawsSan MarinosmSao Tome & PrincipestSaudi ArabiasaSenegalsnSeychellesscSierra LeoneslSingaporesgSlovakiaskSloveniasiSolomon IslandssbSomaliasoSouth AfricazaSouth Georgia and the South Sandwich IslandsgsSpainesSri LankalkSt. HelenashSt. Kitts and NevisknSt. Pierre & MiquelonpmSt. Vincent & the GrenadinesvcSudansdSurinamesrSvalbard & Jan Mayen IslandssjSwazilandszSwedenseSwitzerlandchSyrian Arab RepublicsyTaiwan, Province of ChinatwTajikistantjTanzania, United Republic oftzThailandthTogotgTokelautkTongatoTrinidad & TobagottTunisiatnTurkeytrTurkmenistantmTurks & Caicos IslandstcTuvalutvUgandaugUkraineuaUnion of Soviet Socialist Republics (no longer exists)suUnited Arab EmiratesaeUnited Kingdom (Great Britain)gbUnited States Minor Outlying IslandsumUnited States Virgin IslandsviUnited StatesusUruguayuyUzbekistanuzVanuatuvuVatican City State (Holy See)vaVenezuelaveViet NamvnWallis & Futuna IslandswfWestern SaharaehYemenyeYugoslaviayuZairezrZambiazmZimbabwezw/images/cover.png
/documentation/data-extraction/Data Extraction | ScrapingBee 💡 Important:This page explains how to use a specific feature of our main web scraping API!If you are not yet familiar with ScrapingBee web scraping API, you can read the documentation here. Basic usage If you want to extract data from pages and don't want to parse the HTML on your side, you can add extraction rules to your API call. The simplest way to use extraction rules is to use the following format/images/cover.png
/documentation/fast-search/Fast Search API | ScrapingBee name [type] (default) Description api_key [string] required Your api key Learn more search [string] required The text you would put in the search bar Learn more country_code [string] ("us") Country code used to localize results (ISO 3166-1 alpha-2) Learn more language [string] ("en") Language of the search results (e.g. en, fr) Learn more page [integer] (1) The page number you want to extract results from Learn more Getting Started Our Fast Search API delivers search results in under a second, with a simple, lightweight request./images/cover.png
/documentation/google-api/Google API | ScrapingBee name [type] (default) Description api_key [string] required Your api key Learn more search [string] required The text you would put in the Google search bar Learn more add_html [boolean] (false) Adding the full html of the page in the results Learn more country_code [string] ("us") Country code from which you would like the request to come from Learn more device ["desktop" | "mobile"] ("desktop") Control the device the request will be sent from Learn more extra_params [string] ("") Extra Google URL parameters Learn more language [string] ("en") Language the search results will be displayed in Learn more light_request [boolean] (true) Light requests are faster and cheaper (10 credits instead of 15), but some content may be missing. Learn more nfpr [boolean] (false) Exclude results from auto-corrected queries that were spelt wrong. Learn more page [integer] (1) The page number you want to extract results from Learn more search_type ["classic" | "news" | "maps" | "images" | "lens" | "shopping" | "ai_mode"] ("classic") The type of search you want to perform Learn more Getting Started Our Google Search API allows you to scrape search results pages in realtime./images/cover.png
/documentation/js-scenario/JavaScript Scenario | ScrapingBee 💡 Important:This page explains how to use a specific feature of our main web scraping API!If you are not yet familiar with ScrapingBee web scraping API, you can read the documentation here. Basic usage If you want to interact with pages you want to scrape before we return your the HTML you can add JavaScript scenario to your API call. For example, if you wish to click on a button, you will need to use this scenario./images/cover.png
/documentation/proxy-mode/Proxy Mode | ScrapingBee What is the proxy mode? ScrapingBee also offers a proxy front-end to the API. This can make integration with third-party tools easier. The Proxy mode only changes the way you access ScrapingBee. The ScrapingBee API will then handle requests just like any standard request. Request cost, return code and default parameters will be the same as a standard no-proxy request. We recommend disabling Javascript rendering in proxy mode, which is enabled by default. The following credentials and configurations are used to access the proxy mode:/images/cover.png
/documentation/walmart/Walmart API | ScrapingBee Our Walmart API allows you to scrape Walmart search results and product details in realtime. We provide two endpoints: Search endpoint (/api/v1/walmart/search) - Fetch Walmart search results Product endpoint (/api/v1/walmart/product) - Fetch structured Walmart product details Walmart Search API Quick start To scrape Walmart search results, you only need two things: your API key, available here a search query (learn more about search query) Then, simply do this. Copy curl "https://app.scrapingbee.com/api/v1/walmart/search?api_key=YOUR-API-KEY&query=iphone" # Install the Python Requests library: # `pip install requests` import requests def send_request(): response = requests.get( url="https://app.scrapingbee.com/api/v1/walmart/search", params={ "api_key": "YOUR-API-KEY", "query": "iphone", }, ) print('Response HTTP Status Code: ', response.status_code) print('Response HTTP Response Body: ', response.content) send_request() // request Axios const axios = require('axios'); axios.get('https://app.scrapingbee.com/api/v1/walmart/search', { params: { 'api_key': 'YOUR-API-KEY', 'query': 'iphone', } }).then(function (response) { // handle success console.log(response); }) import java.io.IOException; import org.apache.http.client.fluent.*; public class SendRequest { public static void main(String[] args) { sendRequest(); } private static void sendRequest() { // Classic (GET ) try { // Create request Content content = Request.Get("https://app.scrapingbee.com/api/v1/walmart/search?api_key=YOUR-API-KEY&query=iphone") // Fetch request and return content .execute().returnContent(); // Print content System.out.println(content); } catch (IOException e) { System.out.println(e); } } } require 'net/http' require 'net/https' # Classic (GET ) def send_request uri = URI('https://app.scrapingbee.com/api/v1/walmart/search?api_key=YOUR-API-KEY&query=iphone') # Create client http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_PEER # Create Request req = Net::HTTP::Get.new(uri) # Fetch Request res = http.request(req) puts "Response HTTP Status Code: #{ res.code }" puts "Response HTTP Response Body: #{ res.body }" rescue StandardError => e puts "HTTP Request failed (#{ e.message })" end send_request() <?php // get cURL resource $ch = curl_init(); // set url curl_setopt($ch, CURLOPT_URL, 'https://app.scrapingbee.com/api/v1/walmart/search?api_key=YOUR-API-KEY&query=iphone'); // set method curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET'); // return the transfer as a string curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // send the request and save response to $response $response = curl_exec($ch); // stop if fails if (!$response) { die('Error: "' . curl_error($ch) . '" - Code: ' . curl_errno($ch)); } echo 'HTTP Status Code: ' . curl_getinfo($ch, CURLINFO_HTTP_CODE) . PHP_EOL; echo 'Response Body: ' . $response . PHP_EOL; // close curl resource to free up system resources curl_close($ch); ?> package main import ( "fmt" "io/ioutil" "net/http" ) func sendClassic() { // Create client client := &http.Client{} // Create request req, err := http.NewRequest("GET", "https://app.scrapingbee.com/api/v1/walmart/search?api_key=YOUR-API-KEY&query=iphone", nil) parseFormErr := req.ParseForm() if parseFormErr != nil { fmt.Println(parseFormErr) } // Fetch Request resp, err := client.Do(req) if err != nil { fmt.Println("Failure : ", err) } // Read Response Body respBody, _ := ioutil.ReadAll(resp.Body) // Display Results fmt.Println("response Status : ", resp.Status) fmt.Println("response Headers : ", resp.Header) fmt.Println("response Body : ", string(respBody)) } func main() { sendClassic() } Here is a breakdown of all the parameters you can use with the Walmart Search API:/images/cover.png
/documentation/youtube/YouTube API | ScrapingBee Our YouTube API allows you to scrape YouTube search results, video metadata, transcripts, and trainability information in realtime. We provide four endpoints: Search endpoint (/api/v1/youtube/search) - Fetch YouTube search results Metadata endpoint (/api/v1/youtube/metadata) - Fetch structured YouTube video metadata Transcript endpoint (/api/v1/youtube/transcript) - Fetch YouTube video transcripts Trainability endpoint (/api/v1/youtube/trainability) - Check video transcript availability YouTube Search API Quick start To scrape YouTube search results, you only need two things:/images/cover.png
No rows found, please edit your search term.

Heading structure

Found 11 row(s).
Heading structureCountErrors 🔽URL
  • <h1> HTML API
    • <h2> Overview [#overview]
    • <h2> Getting Started [#getting-started]
    • <h2> API key [#api_key]
    • <h2> URL [#url]
    • <h2> Headless Browser [#headless-browser]
      • <h3> Javascript Rendering [#render_js]
      • <h3> JavaScript Execution [#js_scenario]
      • <h3> Wait for a fixed amount of time [#wait]
      • <h3> Wait for selector [#wait_for]
      • <h3> Wait for browser [#wait_browser]
      • <h3> Blocking Ads [#block_ads]
      • <h3> Blocking Images and CSS [#block_resources]
      • <h3> Viewport width [#window_width]
      • <h3> Viewport height [#window_height]
    • <h2> Proxies [#proxies]
      • <h3> Premium proxy [#premium_proxy]
      • <h3> Geolocation [#country_code]
      • <h3> Stealth Proxy [#stealth_proxy]
      • <h3> Own proxy [#own_proxy]
    • <h2> Headers [#headers]
      • <h3> Header Forwarding [#forward_headers]
      • <h3> Pure Header Forwarding [#forward_headers_pure]
    • <h2> Response format [#response-format]
      • <h3> Downloading Picture and Files [#downloading-picture-and-files]
      • <h3> Data extraction with AI [#data-extraction-with-ai]
      • <h3> AI Query [#ai_query]
      • <h3> AI Selector (Optional) [#ai_selector]
      • <h3> AI data extraction [#ai_extract_rules]
      • <h3> CSS / XPATH data extraction [#extract_rules]
      • <h3> Screenshot [#screenshot]
      • <h3> Screenshot a particular CSS selector [#screenshot_selector]
      • <h3> Screenshot full page [#screenshot_full_page]
      • <h3> JSON Response [#json_response]
      • <h3> Return Page Source [#return_page_source]
      • <h3> ✨ Return Markdown Content [#return_page_markdown]
      • <h3> ✨ Return Text Content [#return_page_text]
      • <h3> Scraping Configuration [#scraping_config]
      • <h3> HTML to PDF conversion [#html-to-pdf-conversion]
    • <h2> Proxy Mode [#proxy-mode]
    • <h2> Session [#session_id]
    • <h2> Timeout (in ms) [#timeout]
    • <h2> Custom Cookies [#cookies]
    • <h2> Device [#device]
    • <h2> Google [#custom_google]
    • <h2> POST / PUT [#post--put]
    • <h2> Credit cost for your requests [#credit-cost-for-your-requests]
    • <h2> Usage endpoint [#usage-endpoint]
    • <h2> Status Code [#status-code]
      • <h3> Response Status Code [#response-status-code]
      • <h3> Transparent HTTP status code [#transparent_status_code]
    • <h2> Response Headers [#response-headers]
520/documentation/
  • <h1> Data Extraction
    • <h2> Basic usage [#basic-usage]
    • <h2> CSS or XPATH selector [#selector_type]
    • <h2> Output Format [#output]
      • <h3> Shortcuts [#shortcuts]
      • <h3> Extracting information from tables [#extracting-information-from-tables]
    • <h2> Single element or list [#type]
    • <h2> Clean Text [#clean]
    • <h2> Extract nested items [#extract-nested-items]
    • <h2> Common use cases [#common-use-cases]
      • <h3> Extract all links from a page [#extract-all-links-from-a-page]
      • <h3> Extract all text from a page [#extract-all-text-from-a-page]
      • <h3> Extract all email addresses from a page [#extract-all-email-addresses-from-a-page]
    • <h2> AI data extraction [#ai-data-extraction]
    • <h2> Short notation [#short-notation]
    • <h2> Advanced notation [#advanced-notation]
      • <h3> Types [#types]
    • <h2> Speed up requests [#speed-up-requests]
180/documentation/data-extraction/
  • <h1> Proxy Mode
    • <h2> What is the proxy mode? [#what-is-the-proxy-mode]
      • <h3> Use proxy mode with your favorite language? [#use-proxy-mode-with-your-favorite-language]
    • <h2> Apify integration [#apify-integration]
    • <h2> Parsehub Integration [#parsehub-integration]
    • <h2> Phantombuster integration [#phantombuster-integration]
    • <h2> Web Scraper integration [#web-scraper-integration]
    • <h2> Kameleo integration [#kameleo-integration]
    • <h2> Octoparse integration [#octoparse-integration]
90/documentation/proxy-mode/
  • <h1> JavaScript Scenario
    • <h2> Basic usage [#basic-usage]
    • <h2> Strict mode [#strict-mode]
    • <h2> Clicking on a button [#click]
    • <h2> Wait for a fixed amount of time [#wait]
    • <h2> Wait for an element to appear [#wait_for]
    • <h2> Wait for an element to appear and click [#wait_for_and_click]
    • <h2> Scroll Horizontally [#scroll_x]
    • <h2> Scroll Vertically [#scroll_y]
    • <h2> Filling form input [#fill]
    • <h2> Infinite scrolling [#infinite_scroll]
    • <h2> Executing custom JavaScript [#evaluate]
    • <h2> Timeout [#timeout]
130/documentation/js-scenario/
  • <h1> Supported Countries
    • <h2> List of supported country codes [#list-of-supported-country-codes]
20/documentation/country_codes/
  • <h1> GPT API
    • <h2> Overview [#overview]
    • <h2> Quick start [#quick-start]
    • <h2> Response Example [#response-example]
      • <h3> API key [#api_key_GPT]
      • <h3> Prompt [#prompt]
    • <h2> Prompt Best Practices [#prompt-best-practices]
    • <h2> Credit cost for your requests [#credit-cost-for-your-requests]
    • <h2> Citations [#citations]
90/documentation/chatgpt/
  • <h1> Google API
    • <h2> Overview [#overview]
    • <h2> Getting Started [#getting-started]
      • <h3> Request Example [#request-example]
      • <h3> Response Example [#response-example]
    • <h2> API key [#api_key]
    • <h2> Light Request [#light_request]
    • <h2> Search [#search]
    • <h2> Search Type [#search_type]
    • <h2> Geolocation [#country_code]
    • <h2> Device [#device]
    • <h2> Page Number [#page]
    • <h2> Autocorrection [#nfpr]
    • <h2> Extra params [#extra_params]
    • <h2> Full HTML [#add_html]
    • <h2> Credit cost for your requests [#credit-cost-for-your-requests]
160/documentation/google-api/
  • <h1> Fast Search API
    • <h2> Overview [#overview]
    • <h2> Getting Started [#getting-started]
      • <h3> Request Example [#request-example]
      • <h3> Response Example [#response-example]
    • <h2> API key [#api_key]
    • <h2> Search [#search]
    • <h2> Page number [#page]
    • <h2> Geolocation [#country_code]
    • <h2> Credit cost for your requests [#credit-cost-for-your-requests]
100/documentation/fast-search/
  • <h1> YouTube API
    • <h2> Overview [#overview]
    • <h2> YouTube Search API [#youtube-search-api]
      • <h3> Quick start [#quick-start]
      • <h3> Response Example [#response-example]
      • <h3> Search Query [#search]
      • <h3> Upload Date [#upload_date]
      • <h3> Result Type [#type]
      • <h3> Duration Filter [#duration]
      • <h3> Sort By [#sort_by]
      • <h3> HD Filter [#hd]
      • <h3> 4K Filter [#4k]
      • <h3> Subtitles Filter [#subtitles]
      • <h3> Creative Commons [#creative_commons]
      • <h3> Live Streams [#live]
      • <h3> 360° Videos [#360]
      • <h3> 3D Videos [#3d]
      • <h3> HDR Videos [#hdr]
      • <h3> Location Filter [#location]
      • <h3> VR180 Videos [#vr180]
    • <h2> YouTube Metadata API [#youtube-metadata-api]
      • <h3> Quick start [#quick-start-1]
      • <h3> Response Example [#response-example-1]
      • <h3> Video ID [#video_id_YouTubeMetadata]
    • <h2> YouTube Transcript API [#youtube-transcript-api]
      • <h3> Quick start [#quick-start-2]
      • <h3> Response Example [#response-example-2]
      • <h3> Video ID [#video_id_YouTubeTranscript]
      • <h3> Language [#language]
      • <h3> Transcript Origin [#transcript_origin]
    • <h2> YouTube Trainability API [#youtube-trainability-api]
      • <h3> Quick start [#quick-start-3]
      • <h3> Response Example [#response-example-3]
      • <h3> Video ID [#video_id_YouTubeTrainability]
    • <h2> Credit cost for your requests [#credit-cost-for-your-requests]
350/documentation/youtube/
  • <h1> Walmart API
    • <h2> Overview [#overview]
    • <h2> Walmart Search API [#walmart-search-api]
      • <h3> Quick start [#quick-start]
      • <h3> Response Example [#response-example]
      • <h3> API key [#api_key_WalmartAPISearch]
      • <h3> Light Request [#light_request_WalmartAPISearch]
      • <h3> Screenshot [#screenshot_WalmartAPISearch]
      • <h3> Start Page [#start_page]
      • <h3> Search Query [#query]
      • <h3> Minimum Price [#min_price]
      • <h3> Maximum Price [#max_price]
      • <h3> Sort By [#sort_by]
      • <h3> Device [#device_WalmartAPISearch]
      • <h3> Domain [#domain_WalmartAPISearch]
      • <h3> Fulfillment Speed [#fulfillment_speed]
      • <h3> Fulfillment Type [#fulfillment_type]
      • <h3> Delivery ZIP [#delivery_zip]
      • <h3> Store ID [#store_id]
      • <h3> Full HTML [#add_html_WalmartAPISearch]
    • <h2> Walmart Product API [#walmart-product-api]
      • <h3> Quick start [#quick-start-1]
      • <h3> Response Example [#response-example-1]
      • <h3> API key [#api_key_WalmartAPIProduct]
      • <h3> Light Request [#light_request_WalmartAPIProduct]
      • <h3> Screenshot [#screenshot_WalmartAPIProduct]
      • <h3> Product ID [#product_id]
      • <h3> Domain [#domain_WalmartAPIProduct]
      • <h3> Device [#device_WalmartAPIProduct]
      • <h3> Delivery ZIP [#delivery_zip_WalmartAPIProduct]
      • <h3> Store ID [#store_id_WalmartAPIProduct]
      • <h3> Full HTML [#add_html_WalmartAPIProduct]
    • <h2> Credit cost for your requests [#credit-cost-for-your-requests]
330/documentation/walmart/
  • <h1> Amazon API
    • <h2> Overview [#overview]
    • <h2> Amazon Product API [#amazon-product-api]
      • <h3> Quick start [#quick-start]
      • <h3> Response Example [#response-example]
      • <h3> API key [#api_key_AmazonProduct]
      • <h3> Light Request [#light_request]
      • <h3> Screenshot [#screenshot_AmazonProduct]
      • <h3> Product ASIN [#query]
      • <h3> Device [#device_AmazonProduct]
      • <h3> Domain [#domain]
      • <h3> Country [#country_AmazonProduct]
      • <h3> ZIP Code [#zip_code_AmazonProduct]
      • <h3> Language [#language]
      • <h3> Currency [#currency]
      • <h3> Full HTML [#add_html_AmazonProduct]
    • <h2> Amazon Search API [#amazon-search-api]
      • <h3> Quick start [#quick-start-1]
      • <h3> Response Example [#response-example-1]
      • <h3> API key [#api_key]
      • <h3> Light Request [#light_request_AmazonSearch]
      • <h3> Screenshot [#screenshot_AmazonSearch]
      • <h3> Search Query [#query_AmazonSearch]
      • <h3> Start Page [#start_page]
      • <h3> Number of Pages [#pages]
      • <h3> Sort By [#sort_by]
      • <h3> Device [#device_AmazonSearch]
      • <h3> Domain [#domain_AmazonSearch]
      • <h3> Country [#country_AmazonSearch]
      • <h3> ZIP Code [#zip_code_AmazonSearch]
      • <h3> Language [#language_AmazonSearch]
      • <h3> Currency [#currency_AmazonSearch]
      • <h3> Category ID [#category_id]
      • <h3> Merchant ID [#merchant_id]
      • <h3> Autoselect Variant [#autoselect_variant]
      • <h3> Full HTML [#add_html_AmazonSearch]
    • <h2> Credit cost for your requests [#credit-cost-for-your-requests]
370/documentation/amazon/
No rows found, please edit your search term.

404 URLs

No 404 URLs found.

Redirected URLs

StatusRedirected URL 🔼Target URLFound at URL
308 /documentation/documentation/
308 /documentation/amazon/documentation/amazon//documentation/
308 /documentation/chatgpt/documentation/chatgpt//documentation/
308 /documentation/fast-search/documentation/fast-search//documentation/
308 /documentation/google-api/documentation/google-api//documentation/
308 /documentation/walmart/documentation/walmart//documentation/
308 /documentation/youtube/documentation/youtube//documentation/

Skipped URLs Summary

ReasonDomainUnique URLs 🔽
Not allowed hostdashboard.scrapingbee.com4
Not allowed hosthelp.scrapingbee.com3
Not allowed hostapp.scrapingbee.com2
Not allowed hosten.wikipedia.org1

Skipped URLs

Found 10 row(s).
ReasonSkipped URL 🔼SourceFound at URL
Not allowed hosthttps://app.scrapingbee.com/account/manage/api_key<a href>/documentation/fast-search/
Not allowed hosthttps://app.scrapingbee.com/account/register<a href>/documentation/fast-search/
Not allowed hosthttps://dashboard.scrapingbee.com/account/login<a href>/documentation/
Not allowed hosthttps://dashboard.scrapingbee.com/account/manage/api_key<a href>/documentation/
Not allowed hosthttps://dashboard.scrapingbee.com/account/register<a href>/documentation/
Not allowed hosthttps://dashboard.scrapingbee.com/dashboard<a href>/documentation/proxy-mode/
Not allowed hosthttps://en.wikipedia.org/wiki/ISO_3166-1<a href>/documentation/
Not allowed hosthttps://help.scrapingbee.com/en/<a href>/documentation/
Not allowed hosthttps://help.scrapingbee.com/en/article/preconfigured-request-settings-1tcst5f/<a href>/documentation/
Not allowed hosthttps://help.scrapingbee.com/en/article/url-encoding-1w5vas8/<a href>/documentation/google-api/
No rows found, please edit your search term.

External URLs

10 external URL(s)
Found 10 row(s).
External URLPages 🔽Found on URL (max 5)
https://app.scrapingbee.com/account/manage/api_key1/documentation/fast-search/
https://app.scrapingbee.com/account/register1/documentation/fast-search/
https://dashboard.scrapingbee.com/account/login1/documentation/
https://dashboard.scrapingbee.com/account/manage/api_key1/documentation/
https://dashboard.scrapingbee.com/account/register1/documentation/
https://dashboard.scrapingbee.com/dashboard1/documentation/proxy-mode/
https://en.wikipedia.org/wiki/ISO_3166-11/documentation/
https://help.scrapingbee.com/en/1/documentation/
https://help.scrapingbee.com/en/article/preconfigured-request-settings-1tcst5f/1/documentation/
https://help.scrapingbee.com/en/article/url-encoding-1w5vas8/1/documentation/google-api/
No rows found, please edit your search term.

TOP fastest URLs

Found 11 row(s).
Time 🔼StatusFast URL
50 ms 200 /documentation/country_codes/
53 ms 200 /documentation/google-api/
55 ms 200 /documentation/proxy-mode/
56 ms 200 /documentation/youtube/
57 ms 200 /documentation/data-extraction/
59 ms 200 /documentation/
60 ms 200 /documentation/js-scenario/
69 ms 200 /documentation/walmart/
71 ms 200 /documentation/chatgpt/
96 ms 200 /documentation/amazon/
96 ms 200 /documentation/fast-search/
No rows found, please edit your search term.

TOP slowest URLs

Found 11 row(s).
Time 🔽StatusSlow URL
96 ms 200 /documentation/fast-search/
96 ms 200 /documentation/amazon/
71 ms 200 /documentation/chatgpt/
69 ms 200 /documentation/walmart/
60 ms 200 /documentation/js-scenario/
59 ms 200 /documentation/
57 ms 200 /documentation/data-extraction/
56 ms 200 /documentation/youtube/
55 ms 200 /documentation/proxy-mode/
53 ms 200 /documentation/google-api/
50 ms 200 /documentation/country_codes/
No rows found, please edit your search term.

Content types

Content typeURLs 🔽Total sizeTotal timeAvg timeStatus 20xStatus 30x
HTML113 MB727 ms66 ms 11 0
Redirect7773 B190 ms27 ms 07

Content types (MIME types)

Content typeURLs 🔽Total sizeTotal timeAvg timeStatus 20xStatus 30x
text/html; charset=utf-8113 MB727 ms66 ms 11 0
text / html7773 B190 ms27 ms 07

Source domains

DomainTotalsHTMLRedirect
www.scrapingbee.com18 / 3MB / 917ms11 / 3MB / 727ms7 / 773B / 190ms

HTTP headers

Found 15 row(s).
Header 🔼OccursUniqueValues previewMin valueMax value
Access-Control-Allow-Origin181*
Alt-Svc181h3=":443"; ma=86400
Cache-Control111public, max-age=0, must-revalidate
Cf-Ray18-[ignored generic values]
Content-Length7-[ignored generic values]0 B0 B
Content-Type182text/html; charset=utf-8 (11) / text/html (7)
Date18-[ignored generic values]2026-03-242026-03-24
Etag10-[ignored generic values]
Link31; rel="preconnect"
Location77[see values below]
Nel181{"report_to":"cf-nel","success_fraction":0.0,"max_age":604800}
Referrer-Policy181strict-origin-when-cross-origin
Report-To1818[see values below]
Server181cloudflare
X-Content-Type-Options111nosniff
No rows found, please edit your search term.

HTTP header values

Found 35 row(s).
HeaderOccursValue
Access-Control-Allow-Origin18*
Alt-Svc18h3=":443"; ma=86400
Cache-Control11public, max-age=0, must-revalidate
Content-Type11text/html; charset=utf-8
Content-Type7text / html
Link3<https://fonts.googleapis.com>; rel="preconnect"
Location1/documentation/chatgpt/
Location1/documentation/
Location1/documentation/fast-search/
Location1/documentation/youtube/
Location1/documentation/amazon/
Location1/documentation/walmart/
Location1/documentation/google-api/
Nel18{"report_to":"cf-nel","success_fraction":0.0,"max_age":604800}
Referrer-Policy18strict-origin-when-cross-origin
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=NheZl7eMoOdD7OfNo7QnuHLooBZevBWm3UTp0jJeQcs15aT%2Bnxy0%2BjJINzjWAGnzlPD3PutZomfvXhGKOL2jGjANGnjZC9l%2FslpwqsTCLH4jE31THUFw6gNj830UHSE%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=MJzORpt0teDsaOoQrvJ%2FdNm%2BYpt1hgsjDoHeCEz1pO4mDXTjBlEqmtCzGPYLP%2FJEooploipaaI0ENimFU1pP8egxva8PZwycsAyppbdk6hjhB4xnLza9Pv6JgEQHy%2F0%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=q8W%2FLzW%2FL4B2x%2FB5Q6OXctaXiAtXvFF3bUFwWBEA6TKBXFbFZ9ssJ0x783ItOvdYOKfa1wS0e2ZiI3kjzerE3IlbfrB47NSjzITnxnfKEVLVLtGxTbauCZFe5j8R2QM%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=j50pHbCwnOViU3WTfM02XisdvbQNuxjJyvvvf325rJTdPQOwJFI6IkVlez60S5pogB%2BcPRuL%2F9j6%2BjR4MX%2F51thkusMwpRLglHkuk6%2BGTlGsDs7kkrTDni7bxS8e6Cw%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=Hi2OgnKz9zXREsXqOCbXVbvwmrEPCMoftG7%2BPr1D8wjxblsXiIUmI9HS4YfgObbZG6xvGidzSFOdt%2FfREQ9IK1ogkK4wdKTfm9gFTyMZKrTk%2Bx3lQtwr%2FO%2FuXX%2BAkv4%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=YrNlxrUgPtRoHm1gxmFcAPh7vBFwTNZcKMjABc1u6cmLWIhpQa7oM1cir3Pu1lJVsG9KfrYltuRBGFTTn%2B5Q1Q00kdjMcwDLTiE4MZUkjms2Qy7v47WAWO1spWP47so%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=xCYD%2F7wWtCfCq1VYO3EKDt7n%2FEW%2BfcMYG7Wnn2VN3UdYk8DjOZJqck%2F7Pz5xeV844215r9vE%2F8TwZvJ4DF%2FWRip%2BkBe3MG7e%2B2Tkh%2F4xdZg0ltmo%2BkhwuBCqu%2BxlyS8%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=7eOLVfyoqytf0GlOFrhklOGnirwms9uLab0cbGA2QldGesaWhzit5tF3%2BSlnp5cVMX2w9ffZXzJHjduvvt7pLiMmYZqFig3hlyFsZepsEuBRliSbuLZFFhlqyLT4Bs4%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=AoAgSBtzxyaT61Uk9sJZay1ULTfxntUJk8v0iJ5wHXdb7%2BC6yWhgeczYE6h3o62eEYnbxZzGTtWEOTiZGH7hWwI5j4Wt1s%2F2bg3sotCuXCLvFUK%2FD8p7wxav3wdK63g%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=k3m940%2FTbyxkBNWL7PZnLs68XgRdNkIwOProbADU3Wd%2FN7Wbpz%2FCvckFnYHDg%2BdV%2Fk6sv2ONpSpIdYsycD6Vp7X09Li%2FsOcKWzQ6dBOz%2BKhbhqh4PH4LtDrsjB5grCM%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=9hQnuN3fOewS6B%2BFaMZb%2FvT1PnE6T8A4DZ8nqiVLWS8%2FN8%2FN8guqbqcYOFzVZzM3BpnbBwbItzeoa4VWaS8ZQsqXjMEM0AWGXZaZi9l1bdq96nCawSR8P1Ub%2BSD1frU%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=Ab3TvbF319zCuFvnhby%2B7ZlRUpefpVrWIW6m6AuhPqdMEW4dv9iNWtKtLdnUdrdTcgHzB0JAXfwJ7yqUpgGH4u0Q%2BfR%2B7fNXsygfwLnU0mv2Gah%2BINLxOcAse1%2FUbCg%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=sqIy4u2KzqhrjfRfOv0UQuB6gyUpR%2BBfED1KYj%2BH%2BOsSR80A43CSWVHGlRcdYAEDKRztTKoojnnS5oQaNjYfFHbPOjuvJPFAxlEKSFSZwHs67JESfDlHRJ8cF0Tytbc%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=BH%2BqGu3VUM3QJ1xSds%2FeA8PvFIoeLqu7dJ8AzwROTXAJ0oOTdJE7r4sS0x4s14Jh15ncQCEut7of1YdGHezIKDATY1YwHaRDzcIkEn1AUskI%2BvGmHPB2IT%2F%2B9RrZp9U%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=v99698lL5JVxvmork7itooo9v4bwbk4pmw5zDBrB%2FEftjLpSOp4SPkFzY4Z3PDT1B%2BJDxeRTsZnVF5Li8UyUuW6%2BBLvGQ5oqRN2HxTn3XHVh9TLc02QEtfQJCjehzDg%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=tYYPM9u%2BU6wic7kv7MATp3ja%2BRb5ZihBXthBC48U2UUSDAL9xPN2wcNX3b07Z3vPDNZbKKpIj%2BKUWeOSp%2FoJIcAo6XRLXsgJYkMHcZ%2F4X9OzujG1scd7tY6q6wV9UDQ%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=R1HjXxoH2GR2w4u3U8P%2FB7PSpjOnWkoTZG4GUiMMoTHAPm6xeXMdu%2FbM2uzAYjdJnFqRcnvNmdB7wm9vWzppOJRFw94PS6mTxqEoeMlrQKHVP1G%2BC7%2BAfBBLwV2z6Ps%3D"}]}
Report-To1{"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=c6bevBQl4vOMOvfUq%2Ftrr8XYBjDX%2FZIKc%2Bt%2F0oaPxsKZWRyOD6yH679djJH%2BErKG0a%2FIvYGyC4BjrYBMWrOzM6xy2KwVcSOi7fzWPPgItodz0YgY3wM2gvTrn0Bzh%2FI%3D"}]}
Server18cloudflare
X-Content-Type-Options11nosniff
No rows found, please edit your search term.

HTTP Caching by content type (only from crawlable domains)

Content typeCache typeURLs 🔽AVG lifetimeMIN lifetimeMAX lifetime
HTMLCache-Control + ETag100 s 0 s 0 s
RedirectNo cache headers7---
HTMLCache-Control10 s 0 s 0 s

HTTP Caching by domain

DomainCache typeURLs 🔽AVG lifetimeMIN lifetimeMAX lifetime
www.scrapingbee.comCache-Control + ETag100 s 0 s 0 s
www.scrapingbee.comNo cache headers7---
www.scrapingbee.comCache-Control10 s 0 s 0 s

HTTP Caching by domain and content type

DomainContent typeCache typeURLs 🔽AVG lifetimeMIN lifetimeMAX lifetime
www.scrapingbee.comHTMLCache-Control + ETag100 s 0 s 0 s
www.scrapingbee.comRedirectNo cache headers7---
www.scrapingbee.comHTMLCache-Control10 s 0 s 0 s

DNS info

Found 10 row(s).
DNS resolving tree
www.scrapingbee.com
  scrapingbee-landing-page.pages.dev
    IPv4: scrapingbee-landing-page.pages.dev.
    IPv4: 188.114.96.3
    IPv4: 188.114.97.3
    IPv6: scrapingbee-landing-page.pages.dev.
    IPv6: 2a06:98c1:3121::3
    IPv6: 2a06:98c1:3120::3
DNS server: 127.0.0.53
No rows found, please edit your search term.

SSL/TLS info

InfoText
IssuerC = US, O = Google Trust Services, CN = WE1
SubjectCN = www.scrapingbee.com
Valid fromFeb 18 04:21:17 2026 GMT (VALID already 34.4 day(s))
Valid toMay 19 05:04:37 2026 GMT (VALID still for 55.6 day(s))
Supported protocolsTLSv1.2, TLSv1.3
RAW certificate outputCertificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            5a:55:51:3c:ee:a0:60:e4:13:d7:60:fd:03:aa:88:f7
        Signature Algorithm: ecdsa-with-SHA256
        Issuer: C = US, O = Google Trust Services, CN = WE1
        Validity
            Not Before: Feb 18 04:21:17 2026 GMT
            Not After : May 19 05:04:37 2026 GMT
        Subject: CN = www.scrapingbee.com
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:da:cd:e3:e6:10:b0:6e:12:4a:ee:34:ea:6a:27:
                    91:63:9a:7f:87:9c:56:e8:f4:59:5f:13:76:be:87:
                    69:63:db:2b:ed:e0:d1:41:a3:ef:8d:97:1e:2d:bd:
                    06:83:87:87:96:09:1e:e6:9f:50:2a:e2:59:03:92:
                    ab:0c:e0:e4:96
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier: 
                DF:EA:55:C1:4F:1A:82:F3:91:F4:D9:14:F8:D8:56:8A:1D:82:7F:A1
            X509v3 Authority Key Identifier: 
                90:77:92:35:67:C4:FF:A8:CC:A9:E6:7B:D9:80:79:7B:CC:93:F9:38
            Authority Information Access: 
                OCSP - URI:http://o.pki.goog/s/we1/WlU
                CA Issuers - URI:http://i.pki.goog/we1.crt
            X509v3 Subject Alternative Name: 
                DNS:www.scrapingbee.com
            X509v3 Certificate Policies: 
                Policy: 2.23.140.1.2.1
            X509v3 CRL Distribution Points: 
                Full Name:
                  URI:http://c.pki.goog/we1/T0VkE8LAbR8.crl
            CT Precertificate SCTs: 
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : 0E:57:94:BC:F3:AE:A9:3E:33:1B:2C:99:07:B3:F7:90:
                                DF:9B:C2:3D:71:32:25:DD:21:A9:25:AC:61:C5:4E:21
                    Timestamp : Feb 18 05:21:18.578 2026 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:44:02:20:5D:EB:E6:80:5D:3A:A4:0B:EE:16:68:1A:
                                AF:56:58:B9:FA:88:CF:E6:39:11:C3:96:8B:78:43:80:
                                0E:C1:5B:AB:02:20:3C:30:E4:B4:D3:6D:7D:9A:E4:28:
                                E5:4D:24:B7:7B:5C:0D:85:3E:E5:83:4B:68:69:C3:3B:
                                C4:9B:96:F6:30:64
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : D1:6E:A9:A5:68:07:7E:66:35:A0:3F:37:A5:DD:BC:03:
                                A5:3C:41:12:14:D4:88:18:F5:E9:31:B3:23:CB:95:04
                    Timestamp : Feb 18 05:21:18.811 2026 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:46:02:21:00:E7:EC:6A:31:16:33:2D:EE:4A:F0:ED:
                                61:5F:93:59:C8:8A:E0:20:4F:7D:15:C3:87:5F:70:59:
                                CC:E0:36:0D:64:02:21:00:A1:2D:0F:6E:52:41:68:1F:
                                38:30:63:0F:67:92:BF:B2:5E:86:76:D3:21:B8:93:BC:
                                9D:07:E0:F0:97:57:A1:33
    Signature Algorithm: ecdsa-with-SHA256
    Signature Value:
        30:44:02:20:14:7f:80:74:42:d9:b2:5b:13:98:47:c1:dd:41:
        59:e3:53:cc:02:01:22:47:07:a9:60:6f:16:31:26:92:18:af:
        02:20:1a:79:c7:e6:4a:43:ab:80:d6:d9:87:b6:de:b6:d5:af:
        bb:e5:87:0d:75:d8:d1:1c:d8:55:f2:bf:58:39:4f:29
RAW protocols output
=== ssl2 ===
s_client: Unknown option: -ssl2
s_client: Use -help for summary.

=== ssl3 ===
s_client: Unknown option: -ssl3
s_client: Use -help for summary.

=== tls1 ===
40D7DFB23F7B0000:error:0A0000BF:SSL routines:tls_setup_handshake:no protocols available:../ssl/statem/statem_lib.c:104:
CONNECTED(00000003)
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 7 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

=== tls1_1 ===
4067BD62AC730000:error:0A0000BF:SSL routines:tls_setup_handshake:no protocols available:../ssl/statem/statem_lib.c:104:
CONNECTED(00000003)
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 7 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

=== tls1_2 ===
depth=2 C = US, O = Google Trust Services LLC, CN = GTS Root R4
verify return:1
depth=1 C = US, O = Google Trust Services, CN = WE1
verify return:1
depth=0 CN = www.scrapingbee.com
verify return:1
CONNECTED(00000003)
---
Certificate chain
 0 s:CN = www.scrapingbee.com
   i:C = US, O = Google Trust Services, CN = WE1
   a:PKEY: id-ecPublicKey, 256 (bit); sigalg: ecdsa-with-SHA256
   v:NotBefore: Feb 18 04:21:17 2026 GMT; NotAfter: May 19 05:04:37 2026 GMT
 1 s:C = US, O = Google Trust Services, CN = WE1
   i:C = US, O = Google Trust Services LLC, CN = GTS Root R4
   a:PKEY: id-ecPublicKey, 256 (bit); sigalg: ecdsa-with-SHA384
   v:NotBefore: Dec 13 09:00:00 2023 GMT; NotAfter: Feb 20 14:00:00 2029 GMT
 2 s:C = US, O = Google Trust Services LLC, CN = GTS Root R4
   i:C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA
   a:PKEY: id-ecPublicKey, 384 (bit); sigalg: RSA-SHA256
   v:NotBefore: Nov 15 03:43:21 2023 GMT; NotAfter: Jan 28 00:00:42 2028 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDpDCCA0ugAwIBAgIQWlVRPO6gYOQT12D9A6qI9zAKBggqhkjOPQQDAjA7MQsw
CQYDVQQGEwJVUzEeMBwGA1UEChMVR29vZ2xlIFRydXN0IFNlcnZpY2VzMQwwCgYD
VQQDEwNXRTEwHhcNMjYwMjE4MDQyMTE3WhcNMjYwNTE5MDUwNDM3WjAeMRwwGgYD
VQQDExN3d3cuc2NyYXBpbmdiZWUuY29tMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcD
QgAE2s3j5hCwbhJK7jTqaieRY5p/h5xW6PRZXxN2vodpY9sr7eDRQaPvjZceLb0G
g4eHlgke5p9QKuJZA5KrDODklqOCAkwwggJIMA4GA1UdDwEB/wQEAwIHgDATBgNV
HSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB0GA1UdDgQWBBTf6lXBTxqC
85H02RT42FaKHYJ/oTAfBgNVHSMEGDAWgBSQd5I1Z8T/qMyp5nvZgHl7zJP5ODBe
BggrBgEFBQcBAQRSMFAwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vLnBraS5nb29nL3Mv
d2UxL1dsVTAlBggrBgEFBQcwAoYZaHR0cDovL2kucGtpLmdvb2cvd2UxLmNydDAe
BgNVHREEFzAVghN3d3cuc2NyYXBpbmdiZWUuY29tMBMGA1UdIAQMMAowCAYGZ4EM
AQIBMDYGA1UdHwQvMC0wK6ApoCeGJWh0dHA6Ly9jLnBraS5nb29nL3dlMS9UMFZr
RThMQWJSOC5jcmwwggEEBgorBgEEAdZ5AgQCBIH1BIHyAPAAdQAOV5S8866pPjMb
LJkHs/eQ35vCPXEyJd0hqSWsYcVOIQAAAZxvMhLyAAAEAwBGMEQCIF3r5oBdOqQL
7hZoGq9WWLn6iM/mORHDlot4Q4AOwVurAiA8MOS00219muQo5U0kt3tcDYU+5YNL
aGnDO8SblvYwZAB3ANFuqaVoB35mNaA/N6XdvAOlPEESFNSIGPXpMbMjy5UEAAAB
nG8yE9sAAAQDAEgwRgIhAOfsajEWMy3uSvDtYV+TWciK4CBPfRXDh19wWczgNg1k
AiEAoS0PblJBaB84MGMPZ5K/sl6GdtMhuJO8nQfg8JdXoTMwCgYIKoZIzj0EAwID
RwAwRAIgFH+AdELZslsTmEfB3UFZ41PMAgEiRwepYG8WMSaSGK8CIBp5x+ZKQ6uA
1tmHtt621a+75YcNddjRHNhV8r9YOU8p
-----END CERTIFICATE-----
subject=CN = www.scrapingbee.com
issuer=C = US, O = Google Trust Services, CN = WE1
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2977 bytes and written 301 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-ECDSA-CHACHA20-POLY1305
Server public key is 256 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-ECDSA-CHACHA20-POLY1305
    Session-ID: 99E5FCAA4ACF1EF76FDEB32B4A6236B969E9BC6CA92708A2A3AFD2EDAEE6F563
    Session-ID-ctx: 
    Master-Key: DA660945BC69FB477751691C834A387A52AFA5A9CC952526C8EAA59B383B06071DBB29F37A547D6404292DF243C32BC4
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 64800 (seconds)
    TLS session ticket:
    0000 - 58 ba fa fe 9f 59 67 54-85 a3 80 be 80 cd cb b7   X....YgT........
    0010 - 17 4f f5 08 18 c0 20 37-9a 03 c3 52 e3 6a 11 2e   .O.... 7...R.j..
    0020 - f9 7c b3 7a b7 c7 34 41-15 0e 0d ca c7 c3 4b 87   .|.z..4A......K.
    0030 - c2 1e 9c 5c 2f c5 f8 5f-e6 c0 d3 55 25 23 8c a6   ...\/.._...U%#..
    0040 - 76 99 7c 25 61 59 c5 c8-73 2a 72 70 83 44 81 a2   v.|%aY..s*rp.D..
    0050 - f6 1f dd 57 83 de 05 3d-53 66 6b d1 e3 94 b0 40   ...W...=Sfk....@
    0060 - bc be 75 94 7c be ee 0f-b5 72 46 6b 60 7a 74 78   ..u.|....rFk`ztx
    0070 - ed 09 35 d8 12 3b 76 fe-ee e6 f8 78 b7 9f 8c 23   ..5..;v....x...#
    0080 - b5 7e 4e f3 7e af 44 6b-70 3b 5c 8f 33 68 47 ac   .~N.~.Dkp;\.3hG.
    0090 - 84 61 b0 d1 22 7f 41 46-2b 3d ad 61 e3 55 ee 9c   .a..".AF+=.a.U..
    00a0 - f2 66 60 7a a2 69 e8 d6-6a 3c 5d cd 0d c4 00 f2   .f`z.i..j<].....
    00b0 - ac 36 64 38 70 32 19 ab-55 24 2b be 31 fa ae 2b   .6d8p2..U$+.1..+

    Start Time: 1774362942
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---
DONE

=== tls1_3 ===
depth=2 C = US, O = Google Trust Services LLC, CN = GTS Root R4
verify return:1
depth=1 C = US, O = Google Trust Services, CN = WE1
verify return:1
depth=0 CN = www.scrapingbee.com
verify return:1
CONNECTED(00000003)
---
Certificate chain
 0 s:CN = www.scrapingbee.com
   i:C = US, O = Google Trust Services, CN = WE1
   a:PKEY: id-ecPublicKey, 256 (bit); sigalg: ecdsa-with-SHA256
   v:NotBefore: Feb 18 04:21:17 2026 GMT; NotAfter: May 19 05:04:37 2026 GMT
 1 s:C = US, O = Google Trust Services, CN = WE1
   i:C = US, O = Google Trust Services LLC, CN = GTS Root R4
   a:PKEY: id-ecPublicKey, 256 (bit); sigalg: ecdsa-with-SHA384
   v:NotBefore: Dec 13 09:00:00 2023 GMT; NotAfter: Feb 20 14:00:00 2029 GMT
 2 s:C = US, O = Google Trust Services LLC, CN = GTS Root R4
   i:C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA
   a:PKEY: id-ecPublicKey, 384 (bit); sigalg: RSA-SHA256
   v:NotBefore: Nov 15 03:43:21 2023 GMT; NotAfter: Jan 28 00:00:42 2028 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDpDCCA0ugAwIBAgIQWlVRPO6gYOQT12D9A6qI9zAKBggqhkjOPQQDAjA7MQsw
CQYDVQQGEwJVUzEeMBwGA1UEChMVR29vZ2xlIFRydXN0IFNlcnZpY2VzMQwwCgYD
VQQDEwNXRTEwHhcNMjYwMjE4MDQyMTE3WhcNMjYwNTE5MDUwNDM3WjAeMRwwGgYD
VQQDExN3d3cuc2NyYXBpbmdiZWUuY29tMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcD
QgAE2s3j5hCwbhJK7jTqaieRY5p/h5xW6PRZXxN2vodpY9sr7eDRQaPvjZceLb0G
g4eHlgke5p9QKuJZA5KrDODklqOCAkwwggJIMA4GA1UdDwEB/wQEAwIHgDATBgNV
HSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB0GA1UdDgQWBBTf6lXBTxqC
85H02RT42FaKHYJ/oTAfBgNVHSMEGDAWgBSQd5I1Z8T/qMyp5nvZgHl7zJP5ODBe
BggrBgEFBQcBAQRSMFAwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vLnBraS5nb29nL3Mv
d2UxL1dsVTAlBggrBgEFBQcwAoYZaHR0cDovL2kucGtpLmdvb2cvd2UxLmNydDAe
BgNVHREEFzAVghN3d3cuc2NyYXBpbmdiZWUuY29tMBMGA1UdIAQMMAowCAYGZ4EM
AQIBMDYGA1UdHwQvMC0wK6ApoCeGJWh0dHA6Ly9jLnBraS5nb29nL3dlMS9UMFZr
RThMQWJSOC5jcmwwggEEBgorBgEEAdZ5AgQCBIH1BIHyAPAAdQAOV5S8866pPjMb
LJkHs/eQ35vCPXEyJd0hqSWsYcVOIQAAAZxvMhLyAAAEAwBGMEQCIF3r5oBdOqQL
7hZoGq9WWLn6iM/mORHDlot4Q4AOwVurAiA8MOS00219muQo5U0kt3tcDYU+5YNL
aGnDO8SblvYwZAB3ANFuqaVoB35mNaA/N6XdvAOlPEESFNSIGPXpMbMjy5UEAAAB
nG8yE9sAAAQDAEgwRgIhAOfsajEWMy3uSvDtYV+TWciK4CBPfRXDh19wWczgNg1k
AiEAoS0PblJBaB84MGMPZ5K/sl6GdtMhuJO8nQfg8JdXoTMwCgYIKoZIzj0EAwID
RwAwRAIgFH+AdELZslsTmEfB3UFZ41PMAgEiRwepYG8WMSaSGK8CIBp5x+ZKQ6uA
1tmHtt621a+75YcNddjRHNhV8r9YOU8p
-----END CERTIFICATE-----
subject=CN = www.scrapingbee.com
issuer=C = US, O = Google Trust Services, CN = WE1
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2823 bytes and written 333 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 256 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
DONE

Crawler stats

Basic stats
Total execution time4.5 s
Total URLs18
Total size3 MB
Requests - total time918 ms
Requests - avg time51 ms
Requests - min time26 ms
Requests - max time97 ms
Requests by status200: 11
308: 7

Analysis stats

Found 21 row(s).
Class::methodExec time 🔽Exec count
SslTlsAnalyzer::getTLSandSSLCertificateInfo667 ms 1
AccessibilityAnalyzer::checkMissingLabels63 ms 11
BestPracticeAnalyzer::checkHeadingStructure57 ms 11
AccessibilityAnalyzer::checkMissingAriaLabels55 ms 11
BestPracticeAnalyzer::checkNonClickablePhoneNumbers53 ms 11
AccessibilityAnalyzer::checkMissingRoles42 ms 11
BestPracticeAnalyzer::checkMaxDOMDepth36 ms 11
AccessibilityAnalyzer::checkMissingLang36 ms 11
BestPracticeAnalyzer::checkInlineSvg11 ms 11
BestPracticeAnalyzer::checkMissingQuotesOnAttributes9 ms 11
AccessibilityAnalyzer::checkImageAltAttributes2 ms 11
SeoAndOpenGraphAnalyzer::analyzeHeadings1 ms 1
SecurityAnalyzer::checkHtmlSecurity1 ms 11
SecurityAnalyzer::checkHeaders0 ms 11
SeoAndOpenGraphAnalyzer::analyzeSeo0 ms 1
SeoAndOpenGraphAnalyzer::analyzeOpenGraph0 ms 1
BestPracticeAnalyzer::checkTitleUniqueness0 ms 1
BestPracticeAnalyzer::checkMetaDescriptionUniqueness0 ms 1
BestPracticeAnalyzer::checkBrotliSupport0 ms 1
BestPracticeAnalyzer::checkAvifSupport0 ms 1
BestPracticeAnalyzer::checkWebpSupport0 ms 1
No rows found, please edit your search term.

Content processor stats

Found 12 row(s).
Class::methodExec time 🔽Exec count
JavaScriptProcessor::findUrls17 ms 11
NextJsProcessor::applyContentChangesBeforeUrlParsing16 ms 11
HtmlProcessor::findUrls14 ms 18
CssProcessor::findUrls3 ms 11
AstroProcessor::findUrls0 ms 11
AstroProcessor::applyContentChangesBeforeUrlParsing0 ms 11
HtmlProcessor::applyContentChangesBeforeUrlParsing0 ms 18
NextJsProcessor::findUrls0 ms 11
JavaScriptProcessor::applyContentChangesBeforeUrlParsing0 ms 11
SvelteProcessor::findUrls0 ms 11
SvelteProcessor::applyContentChangesBeforeUrlParsing0 ms 11
CssProcessor::applyContentChangesBeforeUrlParsing0 ms 11
No rows found, please edit your search term.

Crawler info

Version 2.1.0.20260317
Executed At 2026-03-24 14:35:38
Command siteone-crawler --url=https://www.scrapingbee.com/documentation --markdown-export-dir=/tmp/siteone-scrapingbee --markdown-exclude-selector=header,footer,nav,.sidebar,.menu,.breadcrumb,script,style --timeout=30 --workers=5 --disable-javascript --disable-styles --disable-fonts --disable-images --disable-files --no-color --hide-progress-bar --output=text --include-regex=/documentation/
Hostname ubuntu-8gb-hel1-1
User-Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/26.0.0.0 Safari/537.36 siteone-crawler/2.1.0.20260317