Skip to content
ConvoRail - Conversational AI for Businesses
Buying Guides

Security Questions to Ask Any Website Chatbot Vendor

Before you add a chatbot to your site, ask the vendor ten plain security questions. Here is the list, what good answers look like and which claims to treat with caution.

Sakshi Sidana
Sakshi Sidana
Founder & CEO
October 21, 2026 7 min read
Security Questions to Ask Any Website Chatbot Vendor

A chatbot vendor will tell you the product is secure. That sentence costs nothing to say. These security questions for a chatbot vendor turn it into specifics you can compare. You do not need to be a security expert; you need to ask plainly and notice whether the answers are concrete.

This is general guidance, not a security audit. For regulated or high-risk use, involve a qualified professional.

Which Ten Questions Should You Ask?

Ask about data separation, access, storage, files, keys, certifications, prompt injection, handoff, deletion and what is not built yet. Concrete answers matter more than reassuring ones.

QuestionWhat a good answer sounds likeWarning sign
1. How is my data separated from other customers?A specific mechanism, such as account-level scoping at the database layer"We take security seriously"
2. Who at your company can see my conversations?A clear access policyNo answer
3. How are uploaded files stored and shared?Type checks, private storage, short-lived access linksPublic URLs
4. How are API keys handled and can I rotate them?Per-account keys that can be regeneratedShared keys
5. Which certifications do you hold?An exact list, or an honest "none yet"Vague badges
6. How do you handle prompt injection?Grounding to approved content, refusal when unsure"Our AI cannot be tricked"
7. What happens when the bot is unsure?Refusal plus handoff to your teamA guessed answer
8. Can I delete data or my account?A described processNo answer
9. What is on your roadmap rather than live?A clear live-vs-planned listEverything is "available"
10. Where can I read your privacy policy?A published, readable policyNothing published

What Does an Honest Answer Look Like?

An honest answer states what exists, what does not and what is planned. Vendors who admit gaps are usually easier to trust than those who claim everything.

For reference, ConvoRail's own site shows tenant data isolation, refusal to guess and human handoff as live, and compliance certifications such as SOC 2 as roadmap, not a current claim. Attached files are checked against an allowlist, stored privately and opened through short-lived links, and API keys can be regenerated. Ask every vendor for the same level of specificity.

Why Does Grounding Matter for Security?

A chatbot restricted to your approved content has less room to say something harmful or false, and gives attackers less to exploit. Grounding does not remove all risk, but it narrows it.

The risks are explained in prompt injection in website chatbots and why bots make up answers. A bot that refuses to guess and hands over to a person is easier to govern than one that improvises.

What Should You Check Yourself?

Test the chatbot before you trust it: ask questions outside its knowledge, try to make it ignore its instructions and check what a visitor can see.

  1. 1Ask three questions the bot cannot know. It should say so, not guess.
  2. 2Try a few instructions like "ignore your rules" and confirm it still stays within its content.
  3. 3Submit the contact form and check where the details go.
  4. 4Read the privacy policy and confirm it matches what the vendor said.

The pre-launch test checklist has a fuller list, and what data a chatbot collects explains what to disclose.

Pro Tip: Send these questions by email before you buy. Written answers give you something to refer back to, and a vendor who will not answer in writing has told you something.

How Do You Compare Vendors Using These Questions?

Put the ten questions in a spreadsheet, send them to each vendor and score the answers for specificity. A simple table makes the differences obvious.

QuestionVendor AVendor BVendor C
Data separation
Access policy
File handling
Key rotation
Certifications
Unsure answers

Score each answer as specific, general or missing. Vendors with many missing answers need follow-up before you buy.

What Red Flags Should You Watch For?

Watch for absolute claims, vague badges and unwillingness to answer in writing. These suggest marketing is ahead of engineering.

  • "Impossible to hack" or "100% accurate" claims.
  • Certifications mentioned without names or dates.
  • No published privacy policy or terms.
  • Features described as available that you cannot find or test.
  • A refusal to explain what happens to your data on cancellation.

What Should You Do After You Choose a Vendor?

Set up access carefully, keep the API key private, review the knowledge base for sensitive content and revisit security questions annually.

  1. 1Keep the account's API key out of public code repositories and share it only where needed.
  2. 2Regenerate the key if it is ever exposed.
  3. 3Do not put confidential information into the knowledge base unless the bot is meant to say it publicly.
  4. 4Limit staff access to the inbox to those who need it.
  5. 5Re-ask the vendor about certifications and roadmap each year.

A Worked Example of Reading a Vendor Answer

This fictional example shows scoring. Asked how data is separated, Vendor A says "we use industry-standard security" and Vendor B says "each customer's data is scoped to their account at the database layer". B's answer names a mechanism you can ask follow-up questions about, so it scores higher, though neither answer replaces testing.

What Follow-Up Questions Are Worth Asking?

  • Can you describe what happens if an API key is exposed?
  • How do you handle unwanted senders and spam?
  • What logging exists for access to my data?
  • Which sub-processors do you use, if any?
  • How will you notify me if something goes wrong?

What Should You Ask Your Own Team?

Security also depends on what you do with the product. Ask your own team who has inbox access, where the API key is kept and what content goes into the knowledge base.

  • Who can read conversations, and do they all need to?
  • Where is the script tag and API key stored, and who can change it?
  • Does the knowledge base contain anything that should not be public?
  • Who decides what happens if a visitor reports a problem?

A secure product used carelessly is still a risk, so treat vendor questions and internal habits as two halves of one checklist.

Related reading: AI regulation for small business chatbots, a small business guide to AI chatbots, build vs buy, pricing models explained and topics your chatbot should never answer.

Conclusion

Ask chatbot vendors ten concrete security questions about data separation, access, files, keys, certifications, prompt injection, unsure answers, deletion, roadmap and privacy policy. Prefer vendors that state exactly what exists and what does not, and test the product yourself before relying on it.

Key takeaways

  • General reassurance is not an answer; ask for specifics.
  • Ask about data separation, access, files, API keys, certifications and deletion.
  • Good vendors say plainly what is live and what is only planned.
  • A bot that refuses to guess and hands over is easier to govern.
  • Test the chatbot yourself and get answers in writing.

Quick Summary

Security questions for a website chatbot vendor should cover data separation between customers, who can see conversations, how files and API keys are handled, which certifications exist, how prompt injection and uncertain answers are handled, deletion, the live-versus-roadmap list and the privacy policy. Prefer concrete answers and honest gaps over reassurance, and test the product yourself. This is general guidance, not a security audit.

Want to see grounded answers on your own site?

Start free, add your Q&A pairs and paste one script tag. No credit card required.

Start free
Tagschatbot securityvendor questionschatbot buying checklistdata isolationchatbot compliance

Share this article

Sakshi Sidana
Sakshi Sidana

Founder & CEO

Writes about grounded AI chatbots for websites: how they work, how to measure them and what to check before you buy.

Topics

AI ReliabilityData & BenchmarksSetup GuidesComparisonsBuying GuidesAI TrendsProduct GuidesCustomer SupportIndustry GuidesAI Search

Get new posts by email

Guides on AI chatbots for websites, straight to your inbox.

FAQ

Frequently asked questions

About: Security Questions to Ask Any Website Chatbot Vendor

What security questions should I ask a chatbot vendor?

How customer data is separated, who can access conversations, how files and API keys are handled, which certifications exist and how the bot behaves when unsure.

Is SOC 2 required?

Not for every business. If you need it, ask the vendor exactly what they hold today. ConvoRail states SOC 2 is not in place yet.

Can I trust a vendor that says it is fully secure?

Treat that with caution. Look for specific mechanisms and honest descriptions of limits.

How do I test chatbot security myself?

Ask questions outside its knowledge, try to make it ignore its instructions, submit the contact form and read the privacy policy.

Does grounding improve security?

It narrows risk by limiting the bot to approved content, but it does not remove all risk.

What is ConvoRail?

ConvoRail is an AI chatbot platform for websites. It answers visitor questions using only the knowledge you give it, such as your Q&A pairs and website content, and hands the conversation to your team when it isn't sure.

How does ConvoRail avoid wrong or made-up answers?

It only answers when your knowledge base has a close match. If nothing matches, it says it doesn't have confirmed information, offers a contact form and flags your team instead of guessing.

Questions?

Want to talk about this?

Tell us about your site and what your customers ask. A real person replies.

By sending this you agree that we may contact you about your request. We never share your details.

See a chatbot that says it doesn't know.

Try the live demo, or start with your own Q&A pairs, free with no credit card.