Adding AI to a Ruby on Rails Application: Where to Start

Most teams do not need an "AI strategy" to begin. They need one concrete task that is slow, repetitive and tolerant of an occasional mistake. This article covers how to pick that task and how to wire it into an existing Rails application without creating a new class of production problems.

Start from a task, not a model

The question "which model should we use?" comes too early. Start by writing down a task a person currently does by hand: sorting incoming requests, extracting fields from documents, drafting a first reply, summarizing a long thread. Then ask what happens when the result is wrong. If the answer is "someone corrects it in ten seconds", the task is a good first candidate. If the answer is "money moves", it is not.

Three patterns that cover most first projects

• Document and text processing: classification, extraction, summarization of content you already store.
• Question answering over your own data: retrieve the relevant records, then let a model answer from them.
• Internal workflow automation: a model prepares a draft or a decision, a person confirms it.

Never call a model from a web request

Model calls are slow, variable and occasionally fail. Running them inside a controller ties your response time to someone else's latency. Put them in a background job, with a timeout, retries and a guard so that a retry does not repeat finished work.

class ClassifyTicketJob
  include Sidekiq::Job
  sidekiq_options retry: 5

  def perform(ticket_id)
    ticket = Ticket.find(ticket_id)
    return if ticket.category.present?

    result = LlmClient.complete(
      prompt: TicketPrompt.build(ticket),
      timeout: 20,
      response_schema: TicketPrompt::SCHEMA
    )

    ticket.update!(category: result.fetch("category"), category_source: "model")
  end
end

The early return makes the job safe to retry. The timeout keeps a slow provider from filling your queue. The schema and the strict fetch make the job fail loudly when the answer is not what you expected, and the source column tells you later which values came from a model.

Treat the output as untrusted input

A model answer is text produced by a system you do not control. Validate it against a schema, restrict it to the values your code accepts, and never pass it straight to anything with side effects. For actions that are hard to undo, keep a person in the loop and log who approved what.

Keep control of your data

Decide what is allowed to leave your systems before the first call. Read the provider's terms on retention and training, strip what the task does not need, and record prompts, responses and costs so you can answer questions later. Personal data deserves a deliberate decision, not a default.

Retrieval over your own data

For question answering, you usually do not need a new database. PostgreSQL with the pgvector extension stores embeddings next to the rows they describe, which keeps permissions, backups and migrations in one place. Start with the simplest retrieval that returns the right records, and only add machinery when you can show it is the bottleneck.

Measure with a small test set

Collect a few dozen real examples with the answer a person would give, and run every prompt or model change against them. It takes an afternoon, and it replaces opinions with a number you can track.

If you are weighing where AI fits in your product, a short scoping workshop can save weeks of trying things in the wrong order. See the AI integration page for how we run it.