Getting Started with AI
Part 6 of The Broker's Guide to AI — picking the first tasks, asking vendors the right questions, and rolling out without drama.
Picking the first tasks
Start where three things overlap: high volume, low risk, and easy to verify. In most brokerages that points to the same shortlist:
- Answering internal underwriting questions from carrier manuals, with sources shown.
- Summarizing long documents: policy wordings, quotes, inspection reports.
- Drafting routine client communications for human review.
- Retrieving and sending documents clients request, like proof of insurance.
- Logging activities and preparing renewal summaries.
Avoid starting with anything touching claims, complaints, coverage advice, or binding. Those come later, if ever, and only with a human in control.
Questions to ask every AI vendor
Bring this list to every demo:
- Where do your answers come from — general AI knowledge or our actual documents? Can you show sources?
- What can your AI not do? Show me the guardrails in writing.
- For each workflow, what level of autonomy does it run at, and who controls that?
- What triggers a handoff to a human, and what does the human see when they take over?
- Show me the audit log. Can I reconstruct everything the AI did last Tuesday?
- Where is our data stored? Is it used to train your models for other customers?
- What independent certifications do you hold, and can we see the reports?
- Have you been through a carrier's due-diligence process before?
- What does shadow mode or a pilot look like before anything goes live?
- Who else in insurance is using this in production, and can we talk to them?
Rolling out without drama
The pattern that works, condensed:
- Pick one workflow from the low-risk shortlist. One, not five.
- Run shadow mode for a few weeks. Measure accuracy against what your team would have done.
- Move to draft mode with human approval on everything. Measure time saved.
- Name an owner — someone responsible for the AI's performance, who reviews the logs weekly and decides when a task moves up the trust ladder.
- Tell your team the truth: what the AI does, what it doesn't, what it means for their role, and how to flag problems.
- Expand based on evidence. Add workflows and autonomy only when the numbers support it.
The short version of the entire guide
AI is a fast, well-read new hire. RAG means it answers from your real documents instead of memory. Agents do work instead of just answering questions, and MCP is the standard plug that connects them to your systems. Autonomy comes in levels, and the right level depends on the risk of the task. Humans stay in the loop, with AI earning more independence the way any employee does — by proving itself. In a regulated industry, guardrails, audit trails, explainability, and data protection aren't extras; they're the price of admission. And accountability always stays with you.
