Integration

AgentFi works as a lightweight proxy in front of your site. It intercepts requests from AI bots and serves optimized content (/llms.txt, /sitemap.xml, product pages) straight from your site, while all other traffic passes through unchanged.

Where the proxy sits

Prerequisite: a verified domain#

Deploying the proxy requires verified domain ownership. The procedure is in Domain verification.

Note

Crawling and page optimization are already available without verification. Only the proxy requires it.

Step 1: choose a deployment mode#

All modes are functionally equivalent for AI bots. The differences are in how hard the deployment is, and in the SEO boost for human visitors.

Mode	What it involves	Difference
Cloudflare Worker RECOMMENDED	AgentFi deploys the Worker in your Cloudflare account.	Full feature parity, including JSON-LD and meta tag injection into the `head` for human visitors.
Nginx (self-hosted) LITE MODE	You copy a config block into your own vhost.	AI bots get optimized HTML for any URL; no `head` injection for humans.
Apache (self-hosted) LITE MODE	You copy a config block into your VirtualHost.	AI bots get optimized HTML on the AgentFi paths.

Cloudflare Worker#

Choose between automatic and manual deployment.

Automatic#

Select a saved Cloudflare API token and AgentFi deploys the Worker for you. It is the fastest option. The zone matching your domain is detected automatically.

Add a Cloudflare token
Go to Cloudflare API Tokens → Create Token and use the Edit Cloudflare Workers template. Paste the token into the dashboard. Saved tokens are managed under Account Settings → Cloudflare Tokens.
Confirm the detected zone
AgentFi looks for the zone matching your domain inside that account and shows it.
Click Deploy Worker
The Worker is created and bound to the route on your domain.

Warning

If the panel shows Zone not found, the token belongs to a Cloudflare account that does not manage that domain. The ways out are: add a token from the right account, add the domain to that Cloudflare account, or grant the token Zone Read permission on the domain.

Route conflict

If a Worker route already exists for the same pattern, AgentFi warns about a route conflict and names the worker it is assigned to. Deploying will overwrite that route. Check in your Cloudflare dashboard that the existing worker is not critical before continuing; an explicit confirmation checkbox has to be ticked.

Manual#

If you would rather not share your Cloudflare credentials, download the script and deploy it yourself. The script comes pre-configured with your account ID and settings; no modifications needed.

Download the Worker script from the dashboard.
Deploy it with the Wrangler CLI or by pasting it into the Cloudflare dashboard, and configure the route the panel gives you.
Come back to the dashboard and click Continue to move on to verification.

Self-hosted Nginx and Apache#

AgentFi generates a config block ready to paste into the vhost that serves your domain. The flow has three steps:

Proxy configuration
Copy the generated block into your vhost and reload the server. The panel suggests a file name.
AI agent prompt
AgentFi also generates a prompt you can paste into an agent with SSH access to your server (Claude, ChatGPT, Cursor). The agent installs the configuration, validates it and reloads the server.
Verify & activate
Once the /verify endpoint answers correctly, click the button. The panel includes a Test /verify endpoint button.

Migrating from Cloudflare to self-hosted

If the Cloudflare Worker is still live it will keep intercepting every request at Cloudflare's edge, and your Nginx or Apache configuration will never see the traffic. The zero-downtime migration order is: (1) install the config at the origin and reload; (2) confirm the health URL resolves through your origin; (3) Danger Zone → Disconnect Cloudflare; (4) Verify & Activate.

Step 2: verify and activate#

AgentFi sends a test request to your proxy to confirm it responds correctly, then activates the integration with automatic health monitoring every 10 minutes. The endpoint being checked is shown on screen.

If verification fails, the panel states which URL was checked and offers a link to the troubleshooting guide. Make sure the Worker is deployed and that the route is live on your domain.

Integration status#

Status	Meaning
Integration Active	The proxy answers and the checks pass.
Integration paused	You paused it. The proxy stays out of the way; traffic goes straight to your site.
Integration Warning	The health check failed. Review the time of the last check and the error message.
Not Connected	The proxy has not been deployed yet.

The status pill is also permanently visible in the dashboard's top bar.

Settings#

Client Domain

The hostname that hosts the proxy. Changing it requires verifying the domain again.

Base Path

The path under which AgentFi serves its content.

Outdated edge contract#

When AgentFi updates the edge contract, the panel shows a notice asking you to regenerate and reinstall your proxy configuration. The Regenerate config button produces the new version.

Danger Zone#

Danger

Disconnect Cloudflare removes the Worker from your account and resets the integration. From that moment AI bots receive your original, unoptimized site, and managed files stop being served.

Final check#

With the integration active, verify the result from outside the dashboard:

Open https://your-domain.com/llms.txt — it should return the generated index.
In Pages, use Preview AIO HTML on an optimized page and compare it with View original.
After a few days, check Bot traffic: you should see 200-status visits from GPTBot, ClaudeBot or PerplexityBot.