Skip to content
FunCoding

web-research

Research topics using the web via curl. Use when the user asks to look something up, find current information, or research a topic online.

Star
1,220
Category
Other
License
Apache-2.0
Last commit
Oct 12, 2026

skills/web-research/SKILL.md

View on GitHub

web-research

When to use

User asks to look something up, find current information, research a topic, or answer a question that requires up-to-date knowledge beyond what you have in context.

Common operations

Fetch a web page

curl -sL "https://example.com" | head -200

Search via DuckDuckGo (HTML)

query="your search terms"
curl -sL "https://html.duckduckgo.com/html/?q=$(printf '%s' "$query" | jq -sRr @uri)" \
  | grep -oP 'href="https?://[^"]*"' | head -10

Fetch and extract text from HTML

# Strip HTML tags for readable text
curl -sL "$url" | sed 's/<[^>]*>//g' | sed '/^$/d' | head -100

Fetch JSON APIs

curl -s "https://api.example.com/endpoint" | jq '.results[]'

Download a file

curl -sLO "https://example.com/file.txt"

Tips

  • Always use -s (silent) and -L (follow redirects) with curl
  • Pipe HTML through sed 's/<[^>]*>//g' for rough text extraction
  • Use head or tail to limit output — full web pages are huge
  • For JavaScript-heavy sites, raw curl won't render dynamic content
  • Rate-limit requests — don't hammer endpoints in tight loops
  • Check robots.txt before scraping extensively

Similar Skills

claude-handoff

MAmattpocock/skills288k10,178 new stars in the last 7 days

agent-payment-x402

AFaffaan-m/ECC278k4,046 new stars in the last 7 days

agent-self-evaluation

AFaffaan-m/ECC278k4,046 new stars in the last 7 days

bengali-nlp

AFaffaan-m/ECC278k4,046 new stars in the last 7 days

ponytail-gain

DIDietrichGebert/ponytail162k5,405 new stars in the last 7 days

Featured

ponytail-debt

Scan shortcut comments, build a debt ledger, flag items without an upgrade path.

DIDietrichGebert/ponytail162k5,405 new stars in the last 7 days