A Repository-First Framework for Democratizing Chatbot Development

Abstract

Teams often require chatbots that integrate with business systems; however, existing options involve trade-offs: hosted platforms enable quick setup but impose vendor lock-in, whereas custom frameworks preserve control but exclude non-developers. This study evaluated whether combining a repositoryfirst framework with an AI coding agent can lower development costs and time without reducing reliability or usability. In a within-subjects experiment of eight matched chatbot app microtasks with nine developers and 144 runs, the framework plus agent condition completed tasks 56 percent faster with an agent-to-framework ratio of 2.29 and 95 percent confidence interval from 1.40 to 3.63, and at 43 percent lower cost with an agent-to-framework ratio of 1.76 and 95 percent confidence interval from 1.12 to 2.58 compared to the agent-only condition. Reliability signals improved with fewer human interventions and comparable error rates, token efficiency, and latency. A separate 50-user study reported a System Usability Scale mean of 70.6 with a 95 percent confidence interval from 68.6 to 72.6, exceeding the common benchmark of 68. These results support repositoryfirst, agent-assisted development as a practical approach to democratize chatbot creation while maintaining portability and code ownership.

Ähnliche Arbeiten