Convert URL HTML to Markdown and Get Page Links
Configured with HTTP Request, Limit, Loop Over Items, Split Out, and additional nodes. Review the connections and setup before adapting this template.
Node categories in this workflow
- Trigger
- Wait
- Routing
- Transform
- Web / API
Icons describe general node roles, inferred from node types. Inspect the parameters for the configured action.
Drag to pan; use + and − to zoom. Reconstructed from the exported node positions; nothing runs here. Dashed lines represent non-main connections. Full node names and notes appear below.
How it works
The exported workflow contains the following configured nodes. Their connections are shown above; inspect the JSON for the complete logic, expressions, and external actions.
| Node | Category | Type | Operation |
|---|---|---|---|
| When clicking ‘Test workflow’ | Trigger | Manual Trigger | See node parameters |
| Wait | Wait | Wait | See node parameters |
| Connect to your own data source | Routing | No Op | See node parameters |
| Get urls from own data source | Routing | No Op | See node parameters |
| Example fields from data source | Transform | Edit Fields | See node parameters |
| 40 items at a time | Transform | Limit | See node parameters |
| 10 at a time | Routing | Loop Over Items | See node parameters |
| Markdown data and Links | Transform | Edit Fields | See node parameters |
| Split out page URLs | Transform | Split Out | See node parameters |
| Retrieve Page Markdown and Links | Web / API | HTTP Request | POST |
Setup
- Download the JSON and import it into an isolated n8n instance. Leave the workflow inactive.
- Connect your own credentials through n8n’s credential store. Replace
REPLACE_WITH_*values with your resource identifiers. - Inspect API destinations, Code nodes, database queries, file paths, and outbound actions.
- Run with synthetic data and harmless destinations. Enable triggers only after checking the actual outputs.
Requirements
An n8n instance with the listed node types and compatible versions. Service-connected workflows may require accounts, credentials, or paid APIs.
Credential types declared in the source
httpHeaderAuth
Workflow notes
Sticky Note36
40 at a time seems to be the memory limit on my server - run until complete with batches of 40 or increase based on your server memory
Sticky Note28
Firecrawl.dev retrieves markdown inc. title, description, links & content. First define the URLs you'd like to scrape
Example fields from data source
Define URLs in array
Sticky Note33
REQUIRED
Connect to your database of urls to input. Name the column Page like in the Example fields from data source node and make sure it has one link per row like split out page urls
Sticky Note34
REQUIRED Update the Auth parameter to your own Firecrawl dev token
Header Auth parameter name - Authorization value - your-own-api-key
Sticky Note35
REQUIRED Output the data to your own data source e.g. Airtable
Sticky Note37
Respect API limits (10 requests per min)
Retrieve Page Markdown and Links
curl -X POST https://api.firecrawl.dev/v1/scrape
-H 'Content-Type: application/json'
-H 'CONFIGURE_AUTHENTICATION_IN_N8N'
-d '{
"url": "https://docs.firecrawl.dev",
"formats" : ["markdown", "html"]
}'
Convert URL HTML to Markdown and Get Page Links
Convert URL HTML to Markdown and Get Page Links
Use Case
Transform web pages into AI-friendly markdown format:
- You need to process webpage content for LLM analysis
- You want to extract both content and links from web pages
- You need clean, formatted text without HTML markup
- You want to respect API rate limits while crawling pages
What this Workflow Does
The workflow uses Firecrawl.dev API to process webpages:
- Converts HTML content to markdown format
- Extracts all links from each webpage
- Handles API rate limiting automatically
- Processes URLs in batches from your database
Setup
- Create a Firecrawl.dev account and get your API key
- Add your Firecrawl API key to the HTTP Request node's Authorization header
- Connect your URL database to the input node (column name must be "Page") or edit the array in
Example fields from data source - Configure your preferred output database connection
How to Adjust it to Your Needs
- Modify input source to pull URLs from different databases
- Adjust rate limiting parameters if needed
- Customize output format for your specific use case
Made by Simon @ automake.io
Convert URL HTML to Markdown and Get Page Links
Convert URL HTML to Markdown and Get Page Links
Purpose and structure
This folder contains the Convert URL HTML to Markdown and Get Page Links workflow. Its executable nodes use httpRequest, limit, manualTrigger, noOp, set, splitInBatches, splitOut, wait. The table and connection map below are derived from the JSON, not an inferred guarantee of behavior.
Setup
Import inactive. Configure your own credentials and replace REPLACE_WITH_* values. Review external destinations and Code nodes before running.
Entry points
When clicking ‘Test workflow’
Verification
Sanitized source candidate; not runtime-tested. This overview is generated from node metadata. Read the folder README for the connection map, credential types, and license.
Attribution
https://github.com/Zie619/n8n-workflows/blob/94007c1445d9258a7da116646b79473e7c7c3282/workflows/Wait/0609_Wait_Limit_Import_Webhook.json MIT; preserve source notices.