#### #### ##### #### #### ####### #### ### #### ######### #### ###### #### ###### #### ###################### ##### #### ####### ####### ##### #### ####### ####### # #### ###################### #### #### ###### #### #### #### ## #### #### #### #### ################## #### #### ################## ================================================== # SiteOne Crawler, v2.1.0.20260317 # # Author: jan.reges@siteone.cz # ================================================== Detected terminal width 138 < 140 chars - compact mode activated. URL | Status | Type | Time | Size | Cache | Access. | Best pr. ------------------------------------------------------------------------------------------------------------------------------------ / | 200 | HTML | 180 ms | 42 kB | etag | 3/2 | 1/3 /advanced/lazy-loading/ | 200 | HTML | 179 ms | 43 kB | etag | 3/2 | 1/1/2 /marketplace/ | 200 | HTML | 178 ms | 6 kB | etag | 1/3/2 | 4 /advanced/file-downloading/ | 200 | HTML | 179 ms | 49 kB | etag | 3/2 | 1/3 /advanced/network-console-capture/ | 200 | HTML | 179 ms | 62 kB | etag | 3/2 | 1/3 /core/simple-crawling/ | 200 | HTML | 249 ms | 54 kB | etag | 3/2 | 1/3 /core/self-hosting/ | 200 | HTML | 360 ms | 372 kB | etag | 3/2 | 1/3 /core/deep-crawling/ | 200 | HTML | 236 ms | 159 kB | etag | 3/2 | 1/3 /core/page-interaction/ | 200 | HTML | 223 ms | 86 kB | etag | 3/2 | 1/3 /extraction/clustring-strategies/ | 200 | HTML | 179 ms | 57 kB | etag | 3/2 | 1/3 /core/crawler-result/ | 200 | HTML | 202 ms | 80 kB | etag | 3/2 | 1/3 /core/markdown-generation/ | 200 | HTML | 180 ms | 98 kB | etag | 3/2 | 1/3 /core/content-selection/ | 200 | HTML | 180 ms | 115 kB | etag | 3/2 | 1/3 /advanced/adaptive-strategies/ | 200 | HTML | 179 ms | 77 kB | etag | 3/2 | 1/3 /blog/ | 200 | HTML | 179 ms | 36 kB | etag | 3/2 | 1/3 /api/arun/ | 200 | HTML | 179 ms | 65 kB | etag | 3/2 | 1/3 /api/strategies/ | 200 | HTML | 179 ms | 91 kB | etag | 3/2 | 1/3 /advanced/hooks-auth/ | 200 | HTML | 179 ms | 67 kB | etag | 3/2 | 1/3 /advanced/advanced-features/ | 200 | HTML | 179 ms | 95 kB | etag | 3/2 | 1/3 /core/c4a-script/ | 200 | HTML | 179 ms | 60 kB | etag | 3/2 | 1/3 /core/installation/ | 200 | HTML | 179 ms | 40 kB | etag | 3/2 | 1/3 /api/crawl-result/ | 200 | HTML | 179 ms | 95 kB | etag | 3/2 | 1/3 /advanced/proxy-security/ | 200 | HTML | 179 ms | 78 kB | etag | 3/2 | 1/3 /api/parameters/ | 200 | HTML | 180 ms | 105 kB | etag | 3/2 | 1/1/3 /core/browser-crawler-config/ | 200 | HTML | 180 ms | 87 kB | etag | 3/2 | 1/3 /core/cli/ | 200 | HTML | 179 ms | 61 kB | etag | 3/2 | 1/3 /advanced/session-management/ | 200 | HTML | 179 ms | 74 kB | etag | 3/2 | 1/3 /api/arun_many/ | 200 | HTML | 179 ms | 55 kB | etag | 3/2 | 1/3 /advanced/multi-url-crawling/ | 200 | HTML | 180 ms | 107 kB | etag | 3/2 | 1/3 /branding/ | 200 | HTML | 179 ms | 81 kB | etag | 3/2 | 1/1/3 /advanced/crawl-dispatcher/ | 200 | HTML | 179 ms | 31 kB | etag | 3/2 | 1/3 /extraction/llm-strategies/ | 200 | HTML | 179 ms | 73 kB | etag | 3/2 | 1/3 /core/ask-ai/ | 200 | HTML | 179 ms | 32 kB | etag | 3/2 | 1/1/2 /apps/llmtxt/ | 200 | HTML | 178 ms | 6 kB | etag | 2/2 | 4 /marketplace/admin/ | 200 | HTML | 178 ms | 10 kB | etag | 1/3/2 | 1/3 /core/url-seeding/ | 200 | HTML | 181 ms | 220 kB | etag | 3/2 | 1/3 /core/fit-markdown/ | 200 | HTML | 179 ms | 59 kB | etag | 3/2 | 1/3 /core/local-files/ | 200 | HTML | 179 ms | 60 kB | etag | 3/2 | 1/1/3 /advanced/virtual-scroll/ | 200 | HTML | 179 ms | 69 kB | etag | 3/2 | 1/3 /advanced/ssl-certificate/ | 200 | HTML | 179 ms | 49 kB | etag | 3/2 | 1/3 /extraction/no-llm-strategies/ | 200 | HTML | 180 ms | 163 kB | etag | 3/2 | 1/1/3 /extraction/chunking/ | 200 | HTML | 179 ms | 55 kB | etag | 3/2 | 1/3 /api/async-webcrawler/ | 200 | HTML | 179 ms | 75 kB | etag | 3/2 | 1/3 /core/adaptive-crawling/ | 200 | HTML | 179 ms | 69 kB | etag | 3/2 | 1/3 /api/c4a-script-reference/ | 200 | HTML | 179 ms | 79 kB | etag | 3/2 | 1/3 /advanced/anti-bot-and-fallback/ | 200 | HTML | 179 ms | 67 kB | etag | 3/2 | 1/3 /core/link-media/ | 200 | HTML | 180 ms | 142 kB | etag | 3/2 | 1/3 /apps/c4a-script/ | 200 | HTML | 178 ms | 9 kB | etag | 1/2/3 | 1/1/2 /CONTRIBUTING/ | 200 | HTML | 179 ms | 38 kB | etag | 3/2 | 1/3 /advanced/undetected-browser/ | 200 | HTML | 179 ms | 78 kB | etag | 3/2 | 1/3 /advanced/identity-based-crawling/ | 200 | HTML | 179 ms | 80 kB | etag | 3/2 | 1/3 /core/examples/ | 200 | HTML | 179 ms | 47 kB | etag | 3/2 | 1/3 /apps/ | 200 | HTML | 179 ms | 38 kB | etag | 3/2 | 1/3 /stats/ | 200 | HTML | 179 ms | 48 kB | etag | 3/2 | 1/3 /advanced/pdf-parsing/ | 200 | HTML | 179 ms | 65 kB | etag | 3/2 | 1/3 /core/cache-modes/ | 200 | HTML | 179 ms | 39 kB | etag | 3/2 | 1/3 /core/quickstart/ | 200 | HTML | 179 ms | 101 kB | etag | 3/2 | 1/3 /blog/articles/llm-context-revolution/ | 200 | HTML | 179 ms | 47 kB | etag | 3/2 | 1/3 /blog/articles/adaptive-crawling-revolution/ | 200 | HTML | 179 ms | 58 kB | etag | 3/2 | 1/3 /c4a-script/demo | 404 | HTML | 179 ms | 29 kB | etag | | 1/1/2 /examples/c4a_script/ | 404 | HTML | 179 ms | 29 kB | etag | | 1/1/2 /examples/c4a_script/tutorial/ | 404 | HTML | 179 ms | 29 kB | etag | | 1/1/2 /docs/md_v2/apps/ | 404 | HTML | 179 ms | 29 kB | etag | | 1/1/2 /docs/md_v2/assets/ | 404 | HTML | 179 ms | 29 kB | etag | | 1/1/2 /api/parameters | 301 | Redirect | 178 ms | 147 B | none | | /blog/articles/llm-context-revolution | 301 | Redirect | 178 ms | 191 B | none | | /api/adaptive-crawler/ | 200 | HTML | 179 ms | 54 kB | etag | 3/2 | 1/3 /api/examples/c4a_script/tutorial/ | 404 | HTML | 179 ms | 29 kB | etag | | 1/1/2 /apps/crawl4ai-assistant/ | 200 | HTML | 179 ms | 51 kB | etag | 1/2/3 | 1/3 /core/llmtxt/ | 200 | HTML | 178 ms | 31 kB | etag | 3/2 | 1/1/2 /api/digest/ | 200 | HTML | 179 ms | 49 kB | etag | 3/2 | 1/3 Skipped URLs Summary -------------------- Reason | Domain | Unique URLs ---------------------------------------------------------- Not allowed host | github.com | 14 Not allowed host | discord.gg | 2 Not allowed host | x.com | 2 Not allowed host | www.nstproxy.com | 1 Not allowed host | docs.litellm.ai | 1 Not allowed host | pypi.org | 1 Not allowed host | trendshift.io | 1 Not allowed host | twitter.com | 1 Not allowed host | www.capsolver.com | 1 Not allowed host | pepy.tech | 1 Not allowed host | forms.gle | 1 Not allowed host | www.linkedin.com | 1 Not allowed host | badge.fury.io | 1 Not allowed host | developer.mozilla.org | 1 Skipped URLs ------------ Reason | Skipped URL | Source | Found at URL ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Not allowed host | https://badge.fury.io/py/crawl4ai | | / Not allowed host | https://developer.mozilla.org/en-US/docs…ts/Using_shadow_DOM | | /core/content-selection/ Not allowed host | https://discord.gg/crawl4ai | | /core/self-hosting/ Not allowed host | https://discord.gg/jP8KfhDhyN | | / Not allowed host | https://docs.litellm.ai/docs/providers | | /core/cli/ Not allowed host | https://forms.gle/E9MyPaNXACnAMaqG7 | | / Not allowed host | https://github.com/BerriAI/litellm | | /extraction/llm-strategies/ Not allowed host | https://github.com/sponsors/unclecode | | / Not allowed host | https://github.com/unclecode | | /apps/crawl4ai-assistant/ Not allowed host | https://github.com/unclecode/crawl4ai | | / Not allowed host | https://github.com/unclecode/crawl4ai/blob/main/LICENSE | | / Not allowed host | https://github.com/unclecode/crawl4ai/bl…/adaptive_crawling/ | | /core/examples/ Not allowed host | https://github.com/unclecode/crawl4ai/bl…xamples/c4a_script/ | | /core/c4a-script/ Not allowed host | https://github.com/unclecode/crawl4ai/issues | | /core/self-hosting/ Not allowed host | https://github.com/unclecode/crawl4ai/network/members | | / Not allowed host | https://github.com/unclecode/crawl4ai/stargazers | | / Not allowed host | https://github.com/unclecode/crawl4ai/tr…s/adaptive_crawling | | /core/adaptive-crawling/ Not allowed host | https://github.com/unclecode/crawl4ai/tr…ver_captcha_solver/ | | /core/examples/ Not allowed host | https://github.com/unclecode/crawl4ai/tr…docs/examples/proxy | | /core/examples/ Not allowed host | https://github.com/unclecode/crawl4ai/tr…es/undetectability/ | | /core/examples/ Not allowed host | https://pepy.tech/project/crawl4ai | | / Not allowed host | https://pypi.org/project/crawl4ai/ | | / Not allowed host | https://trendshift.io/repositories/11716 | | / Not allowed host | https://twitter.com/unclecode | | /blog/ Not allowed host | https://www.capsolver.com/?utm_source=cr…rawl4ai_integration | | /core/examples/ Not allowed host | https://www.linkedin.com/company/crawl4ai | | / Not allowed host | https://www.nstproxy.com/?utm_source=crawl4ai | | /core/examples/ Not allowed host | https://x.com/crawl4ai | | / Not allowed host | https://x.com/unclecode | | /blog/articles/adaptive-crawling-revolution/ External URLs ------------- External URL | Pages | Found on URL (max 5) ------------------------------------------------------------------------------------------------------------------------------------- https://badge.fury.io/py/crawl4ai | 1 | / https://developer.mozilla.org/en-US/docs…ts/Using_shadow_DOM | 1 | /core/content-selection/ https://discord.gg/crawl4ai | 1 | /core/self-hosting/ https://discord.gg/jP8KfhDhyN | 1 | / https://docs.litellm.ai/docs/providers | 1 | /core/cli/ https://forms.gle/E9MyPaNXACnAMaqG7 | 1 | / https://github.com/BerriAI/litellm | 1 | /extraction/llm-strategies/ https://github.com/sponsors/unclecode | 1 | / https://github.com/unclecode | 1 | /apps/crawl4ai-assistant/ https://github.com/unclecode/crawl4ai | 1 | / https://github.com/unclecode/crawl4ai/blob/main/LICENSE | 1 | / https://github.com/unclecode/crawl4ai/bl…/adaptive_crawling/ | 1 | /core/examples/ https://github.com/unclecode/crawl4ai/bl…xamples/c4a_script/ | 1 | /core/c4a-script/ https://github.com/unclecode/crawl4ai/issues | 1 | /core/self-hosting/ https://github.com/unclecode/crawl4ai/network/members | 1 | / https://github.com/unclecode/crawl4ai/stargazers | 1 | / https://github.com/unclecode/crawl4ai/tr…s/adaptive_crawling | 1 | /core/adaptive-crawling/ https://github.com/unclecode/crawl4ai/tr…ver_captcha_solver/ | 1 | /core/examples/ https://github.com/unclecode/crawl4ai/tr…docs/examples/proxy | 1 | /core/examples/ https://github.com/unclecode/crawl4ai/tr…es/undetectability/ | 1 | /core/examples/ https://pepy.tech/project/crawl4ai | 1 | / https://pypi.org/project/crawl4ai/ | 1 | / https://trendshift.io/repositories/11716 | 1 | / https://twitter.com/unclecode | 1 | /blog/ https://www.capsolver.com/?utm_source=cr…rawl4ai_integration | 1 | /core/examples/ https://www.linkedin.com/company/crawl4ai | 1 | / https://www.nstproxy.com/?utm_source=crawl4ai | 1 | /core/examples/ https://x.com/crawl4ai | 1 | / https://x.com/unclecode | 1 | /blog/articles/adaptive-crawling-revolution/ Redirected URLs --------------- Status | Redirected URL | Target URL | Found at URL -------------------------------------------------------------------------------------------------------------------------------------- 301 | /api/parameters | /api/parameters/ | /extraction/llm-strategies/ 301 | /blog/articles/llm-context-revolution | /blog/articles/llm-context-revolution/ | /apps/llmtxt/ 404 URLs -------- Status | URL 404 | Found at URL ---------------------------------------------------------------------------------------------------------------------------------------- 404 | /api/examples/c4a_script/tutorial/ | /api/c4a-script-reference/ 404 | /c4a-script/demo | /core/c4a-script/ 404 | /docs/md_v2/apps/ | /branding/ 404 | /docs/md_v2/assets/ | /branding/ 404 | /examples/c4a_script/ | /core/c4a-script/ 404 | /examples/c4a_script/tutorial/ | /core/c4a-script/ SSL/TLS info ------------ Info | Text --------------------------------------------------------------------------------------------------------------------------------------- Issuer | C = US, O = Let's Encrypt, CN = E8 Subject | CN = crawl4ai.com Valid from | Mar 10 12:36:46 2026 GMT (VALID already 14.1 day(s)) Valid to | Jun 8 12:36:45 2026 GMT (VALID still for 75.9 day(s)) Supported protocols | TLSv1.2, TLSv1.3 RAW certificate output | Certificate: Data: Version: 3 (0x2) Serial Number:…4:ef:d9:2a:f7:86:1c:78:b9:92:b6:83 RAW protocols output | === ssl2 === s_client: Unknown option: -ssl2 s_client: Use -help for su…erify return code: 0 (ok) --- DONE TOP fastest URLs ---------------- Time | Status | Fast URL ------------------------------------------------------------------------------------------------------------------------------------------ 178 ms | 200 | /marketplace/admin/ 178 ms | 200 | /apps/c4a-script/ 178 ms | 200 | /apps/llmtxt/ 178 ms | 200 | /marketplace/ 179 ms | 200 | /core/llmtxt/ 179 ms | 200 | /core/ask-ai/ 179 ms | 200 | /advanced/crawl-dispatcher/ 179 ms | 200 | /advanced/lazy-loading/ 179 ms | 200 | /blog/ 179 ms | 200 | /core/examples/ 179 ms | 200 | /core/installation/ 179 ms | 200 | /advanced/file-downloading/ 179 ms | 200 | /core/cache-modes/ 179 ms | 200 | /CONTRIBUTING/ 179 ms | 200 | /apps/ 179 ms | 200 | /api/arun_many/ 179 ms | 200 | /api/arun/ 179 ms | 200 | /advanced/ssl-certificate/ 179 ms | 200 | /stats/ 179 ms | 200 | /blog/articles/adaptive-crawling-revolution/ TOP slowest URLs ---------------- Time | Status | Slow URL ------------------------------------------------------------------------------------------------------------------------------------- 360 ms | 200 | /core/self-hosting/ 249 ms | 200 | /core/simple-crawling/ 236 ms | 200 | /core/deep-crawling/ 223 ms | 200 | /core/page-interaction/ 202 ms | 200 | /core/crawler-result/ 181 ms | 200 | /core/url-seeding/ 180 ms | 200 | /extraction/no-llm-strategies/ 180 ms | 200 | / 180 ms | 200 | /core/link-media/ 180 ms | 200 | /advanced/multi-url-crawling/ 180 ms | 200 | /api/parameters/ 180 ms | 200 | /core/content-selection/ 180 ms | 200 | /core/markdown-generation/ 180 ms | 200 | /core/browser-crawler-config/ 180 ms | 200 | /core/quickstart/ 179 ms | 200 | /advanced/advanced-features/ 179 ms | 200 | /advanced/pdf-parsing/ 179 ms | 200 | /advanced/identity-based-crawling/ 179 ms | 200 | /api/crawl-result/ 179 ms | 200 | /extraction/clustring-strategies/ SEO metadata ------------ This table contains large data and shows max 10 rows. To see them all, use output to HTML using `--output-html-report=tmp/myreport.html`. URL | Indexing | Title | H1 | Description | Keywords --------------------------------------------------------------------------------------------------------------------------------------- / | Allowed | Home - C….x) | 🚀🤖 Crawl…per | 🚀🤖 Crawl…per | /CONTRIBUTING/ | Allowed | Contribu….x) | Contribu…4AI | 🚀🤖 Crawl…per | /advanced/adaptive-strategies/ | Allowed | Adaptive….x) | Advanced…ies | 🚀🤖 Crawl…per | /advanced/advanced-features/ | Allowed | Overview….x) | Overview…res | 🚀🤖 Crawl…per | /advanced/anti-bot-and-fallback/ | Allowed | Anti-Bot….x) | Anti-Bot…ack | 🚀🤖 Crawl…per | /advanced/crawl-dispatcher/ | Allowed | Crawl Di….x) | Crawl Di…her | 🚀🤖 Crawl…per | /advanced/file-downloading/ | Allowed | File Dow….x) | Download…4AI | 🚀🤖 Crawl…per | /advanced/hooks-auth/ | Allowed | Hooks &….x) | Hooks &…ler | 🚀🤖 Crawl…per | /advanced/identity-based-crawling/ | Allowed | Identity….x) | Preserve…4AI | 🚀🤖 Crawl…per | /advanced/lazy-loading/ | Allowed | Lazy Loa….x) | Missing H1 | 🚀🤖 Crawl…per | OpenGraph metadata ------------------ No URLs with OpenGraph data (og:* or twitter:* meta tags). Heading structure ----------------- This table contains large data and shows max 10 rows. To see them all, use output to HTML using `--output-html-report=tmp/myreport.html`. Heading structure | Count | Errors | URL ----------------------------------------------------------------------------------------------------------------------------------------

Extracting JSON (No LLM) [#extracting-json-no-llm]…Conclusion [#11-conclusion] | 36 | 5 | /extraction/no-llm-strategies/

Handling Lazy-Loaded Images [#handling-lazy-loaded-…ing [#tips-troubleshooting] | 4 | 4 | /advanced/lazy-loading/

Welcome to C4A-Script Tutorial!

C4A-Script Edi…ng Timeline

Playground | 4 | 4 | /apps/c4a-script/

Crawl4AI Assistant

You don't need Puppeteer. Y…

Smart Field Detection | 20 | 4 | /apps/crawl4ai-assistant/

1. BrowserConfig – Controlling the Browser [#1-brow…#4-putting-it-all-together] | 24 | 3 | /api/parameters/

🎨 Crawl4AI Brand Book [#crawl4ai-brand-book]

C…ns]

🎨 Keep It Terminal | 42 | 3 | /branding/

Prefix-Based Input Handling in Crawl4AI [#prefix-ba…1> Conclusion [#conclusion] | 6 | 3 | /core/local-files/

Chunking Strategies [#chunking-strategies]

Why…ing-with-cosine-similarity] | 4 | 3 | /extraction/chunking/

[ Admin Access ]

[ Admin Dashboard ]

Dash…h2> Add/Edit [#modal-title] | 9 | 2 | /marketplace/admin/

🚀🤖 Crawl4AI: Open-Source LLM-Friendly Web Crawler &…Quick Links [#quick-links] | 10 | 0 | / HTTP headers ------------ Header | Occurs | Unique | Values preview | Min value | Max value --------------------------------------------------------------------------------------------------------------- Content-Length | 2 | - | [ignored generic values] | 178 B | 178 B Content-Type | 71 | 1 | text/html | | Date | 71 | - | [ignored generic values] | 2026-03-24 | 2026-03-24 Etag | 69 | - | [ignored generic values] | | Last-Modified | 63 | - | [ignored generic values] | 2026-02-24 | 2026-02-24 Location | 2 | 2 | /blog/articles/llm-context-revol…parameters/ (1) | | Server | 71 | 1 | nginx/1.24.0 (Ubuntu) | | HTTP header values ------------------ Header | Occurs | Value ------------------------------------------------------------------------------------------------------------ Content-Type | 71 | text/html Location | 1 | /blog/articles/llm-context-revolution/ Location | 1 | /api/parameters/ Server | 71 | nginx/1.24.0 (Ubuntu) HTTP Caching by content type (only from crawlable domains) ---------------------------------------------------------- Content type | Cache type | URLs | AVG lifetime | MIN lifetime | MAX lifetime ---------------------------------------------------------------------------- HTML | ETag + Last-Modified | 63 | - | - | - HTML | ETag | 6 | - | - | - Redirect | No cache headers | 2 | - | - | - HTTP Caching by domain ---------------------- Domain | Cache type | URLs | AVG lifetime | MIN lifetime | MAX lifetime ------------------------------------------------------------------------------------ docs.crawl4ai.com | ETag + Last-Modified | 63 | - | - | - docs.crawl4ai.com | ETag | 6 | - | - | - docs.crawl4ai.com | No cache headers | 2 | - | - | - HTTP Caching by domain and content type --------------------------------------- Domain | Content type | Cache type | URLs | AVG lifetime | MIN lifetime | MAX lifetime --------------------------------------------------------------------------------------------------- docs.crawl4ai.com | HTML | ETag + Last-Modified | 63 | - | - | - docs.crawl4ai.com | HTML | ETag | 6 | - | - | - docs.crawl4ai.com | Redirect | No cache headers | 2 | - | - | - TOP non-unique titles --------------------- Count | Title ------------------------------------------------------------------------------------------------------------------------------------------ 2 | Browser, Crawler & LLM Config - Crawl4AI Documentation (v0.8.x) TOP non-unique descriptions --------------------------- Count | Description ------------------------------------------------------------------------------------------------------------------------------------------ 58 | 🚀🤖 Crawl4AI, Open-source LLM-Friendly Web Crawler & Scraper 5 | Best practices -------------- Analysis name | OK | Notice | Warning | Critical ---------------------------------------------------------------------- DOM depth (> 30) | 69 | 0 | 0 | 0 Heading structure | 61 | 0 | 68 | 5 Title uniqueness (> 10%) | 62 | 0 | 0 | 0 Description uniqueness (> 10%) | 1 | 0 | 1 | 0 Brotli support | 0 | 0 | 63 | 0 WebP support | 0 | 0 | 1 | 0 AVIF support | 0 | 0 | 1 | 0 Accessibility ------------- Analysis name | OK | Notice | Warning | Critical -------------------------------------------------------------------- Missing aria labels | 1 | 0 | 2980 | 14 Missing html lang attribute | 1 | 0 | 0 | 0 Missing form labels | 0 | 0 | 7 | 0 Missing image alt attributes | 17 | 0 | 0 | 0 Missing roles | 0 | 0 | 8 | 0 Source domains -------------- Domain | Totals | HTML | Redirect ------------------------------------------------------------ docs.crawl4ai.com | 71/5MB/13s | 69/5MB/12s | 2/338B/357ms Content types ------------- Content type | URLs | Total size | Total time | Avg time | Status 20x | Status 30x | Status 40x -------------------------------------------------------------------------------------------------- HTML | 69 | 5 MB | 12 s | 184 ms | 63 | 0 | 6 Redirect | 2 | 338 B | 357 ms | 178 ms | 0 | 2 | 0 Content types (MIME types) -------------------------- Content type | URLs | Total size | Total time | Avg time | Status 20x | Status 30x | Status 40x ---------------------------------------------------------------------------------------------------------------- text/html | 71 | 5 MB | 13 s | 184 ms | 63 | 2 | 6 DNS info -------- DNS resolving tree ------------------------------------------------------------------------ docs.crawl4ai.com IPv4: 35.163.245.47 DNS server: 127.0.0.53 Security -------- Header | OK | Notice | Warning | Critical | Recommendation ----------------------------------------------------------------------------------------------------------------------------------------- Strict-Transport-Security | 0 | 0 | 0 | 69 | Strict-Transport-Security header is not set. I…against MITM attacks. Content-Security-Policy | 0 | 0 | 0 | 69 | Content-Security-Policy header is not set. It…prevents XSS attacks. Server | 0 | 0 | 0 | 69 | Server header is set to 'nginx/1.24.0 (Ubuntu)…ially their versions. X-Frame-Options | 0 | 0 | 69 | 0 | X-Frame-Options header is not set. It prevents…deny' or 'sameorigin. X-Content-Type-Options | 0 | 0 | 69 | 0 | X-Content-Type-Options header is not set. It s…content type attacks. Referrer-Policy | 0 | 0 | 69 | 0 | Referrer-Policy header is not set. It controls…privacy and security. Feature-Policy | 0 | 0 | 69 | 0 | Feature-Policy header is not set. It allows en…ssions-Policy is set. Permissions-Policy | 0 | 0 | 69 | 0 | Permissions-Policy header is not set. It allow…eatures for security. X-XSS-Protection | 69 | 0 | 0 | 0 | Analysis stats -------------- Class::method | Exec time | Exec count ------------------------------------------------------------------------------- SslTlsAnalyzer::getTLSandSSLCertificateInfo | 3.1 s | 1 AccessibilityAnalyzer::checkMissingAriaLabels | 289 ms | 63 AccessibilityAnalyzer::checkMissingLabels | 265 ms | 63 BestPracticeAnalyzer::checkHeadingStructure | 233 ms | 69 AccessibilityAnalyzer::checkMissingRoles | 225 ms | 63 AccessibilityAnalyzer::checkMissingLang | 199 ms | 63 BestPracticeAnalyzer::checkMaxDOMDepth | 194 ms | 69 BestPracticeAnalyzer::checkNonClickablePhoneNumbers | 73 ms | 69 BestPracticeAnalyzer::checkMissingQuotesOnAttributes | 15 ms | 69 SeoAndOpenGraphAnalyzer::analyzeHeadings | 8 ms | 1 SecurityAnalyzer::checkHtmlSecurity | 8 ms | 69 AccessibilityAnalyzer::checkImageAltAttributes | 5 ms | 63 BestPracticeAnalyzer::checkInlineSvg | 5 ms | 69 SecurityAnalyzer::checkHeaders | 1 ms | 69 SeoAndOpenGraphAnalyzer::analyzeSeo | 0 ms | 1 BestPracticeAnalyzer::checkTitleUniqueness | 0 ms | 1 BestPracticeAnalyzer::checkMetaDescriptionUniqueness | 0 ms | 1 SeoAndOpenGraphAnalyzer::analyzeOpenGraph | 0 ms | 1 BestPracticeAnalyzer::checkBrotliSupport | 0 ms | 1 BestPracticeAnalyzer::checkWebpSupport | 0 ms | 1 BestPracticeAnalyzer::checkAvifSupport | 0 ms | 1 Content processor stats ----------------------- Class::method | Exec time | Exec count ----------------------------------------------------------------------------------- HtmlProcessor::findUrls | 89 ms | 71 NextJsProcessor::applyContentChangesBeforeUrlParsing | 36 ms | 69 JavaScriptProcessor::findUrls | 34 ms | 69 CssProcessor::findUrls | 3 ms | 69 AstroProcessor::findUrls | 0 ms | 69 AstroProcessor::applyContentChangesBeforeUrlParsing | 0 ms | 69 NextJsProcessor::findUrls | 0 ms | 69 JavaScriptProcessor::applyContentChangesBeforeUrlParsing | 0 ms | 69 HtmlProcessor::applyContentChangesBeforeUrlParsing | 0 ms | 71 SvelteProcessor::findUrls | 0 ms | 69 SvelteProcessor::applyContentChangesBeforeUrlParsing | 0 ms | 69 CssProcessor::applyContentChangesBeforeUrlParsing | 0 ms | 69