ITS Ask Us ChatBot

An AI ChatBot that answers ITS related questions.

Project maintained by darkmode-askus Hosted on GitHub Pages — Theme by mattgraham

Table of contents

Ask Hoku

Team DarkMode - ITS AskUs (HACC)

Overview

Problem: The current UH ITS AskUs webpage has an outdated search system which makes it difficult for users to find the answers that they are looking for. Simple searches like “How do I connect to the wifi?”, gives out a list of articles for the user to look through. This can be very time consuming for users to use just to find an answer to a simple question. Because of the non-user friendly design of the current ITS website, many users go to a Help Desk assistant instead of using the current Ask Us search bar.

Solution: Our team will create a ChatBot with the use of modern AI tools that will improve on the current search functionality. This tool will allow users to find the answers they need using conversational language without the need for a Help Desk assistant.

Screenshot 2023-11-15 at 11 32 05 PM

User’s Guide

This section provides a walkthrough of the user interface and its capabilities.

Landing

The landing page is presented to users and is what they see when they first load the application. Here users are able to receive announcements and ITS resources that they can access.

Log In Page

The user needs to login in order to use the chatbot and report any messages from the chatbot. Admins can also login to view the FAQ or report page and make any modifications.

Chatbot

Users can ask the chatbot any questions relating to ITS. The chatbot will produce an answer from the ITS documents and send it to the user.

FAQ Page

All of the frequently asked questions and answers will be posted on this page. Answers will be provided by the admin. Admins have the ability to edit any existing FAQs or add a new one.

Forum Page

Users will able to post their experiences or issues with the chatbot. The users can also post additional ITS questions that require more assistance. These posts can be answered by other users, faculty, or admins at UH. Admins have the additional capability to delete or modify any posts.

Report Page

Here users are able to report any issues or inaccurate answers with the AI chatbot. Admins can see these reports and have the choice to resolve them or simply delete it.

Community Feedback

The feedback on AskHoku was generally positive regarding the design, user experience, and chatbot responses to ITS-related questions. Some users appreciated Hoku’s creative design and the website’s easy navigation. Users found it impressive how Hoku understands conversational language and provides clear, concise responses. However, there are suggestions for improving our community forum page. Some users recommended implementing a ‘like’ system and filters for easier navigation of forum posts. Additionally, users wanted the ability to save their favorite forum posts to their accounts for future reference. Overall, while users value Hoku’s helpfulness and intuitive design, some suggest enhancing the forum’s functionality and user experience.

Feedback Comments:

We are interested in your experience using Ask Hoku! Tell us any feedback you have for us by taking a couple of minutes to fill out the Ask Hoku Feedback Form. It contains only five short questions and will help us understand how to improve the system.

Developer’s Guide

This section will guide Meteor developers through the process of downloading, installing, and running for those who wish to use our code and modify the system.

Installation

First, install Meteor.

Second, visit the Ask Hoku application github page, and click the “Use this template” button to make a copy of this application and create your own repository. Then download the copy of the repo to your local computer.

Third, cd into the darkmode-askus/app directory and install libraries using:

$ meteor npm install

Fourth, run the system with:

$ meteor npm run start

The application will appear at http://localhost:3000 if all goes well.

Reset project

If you wish to reset the project, run the following commands:

$ cd app
$ meteor reset
$ meteor npm run start

Create Meteor settings directory

To create the Meteor settings directory, run the following commands:

$ cd app
$ mkdir config
$ cd config
$ touch settings.development.json

settings.development.json

The settings.development.json file should look similar to this:

{
  "private": {
    "openai-api-key": "api-key",

    "accounts": [
      { "email": "email", "password": "password", "role": "admin" },
      { "email": "email", "password": "password" }
    ]
  }
}

Python venv setup

To create the Python venv, run the following commands:

$ cd data-extraction
$ python3 -m venv venv
$ source venv/bin/activate
$ pip3 install -r requirements.txt

Starting Flask server

To start the Flask server, run the following commands:

$ cd api
$ python main.py

Continuous Integration

darkmode-askus

Ask Hoku uses GitHub Actions to automatically run and check for any errors each time a commit is made to the default branch. You can see the results of all recent “workflows” at https://github.com/darkmode-askus/darkmode-askus/actions.

The workflow definition file is located at .github/workflows/ci.yml and is easy to implement.

Diagram

The diagram below shows how both the front and backend work hand in hand with each other to give users the best experience possible. It also shows how Hoku proccesses the user’s input and how it responds back to the user.

Security Considerations

A prompt injection is a type of cyberattack on an AI system designed to enable the user to perform unauthorized actions. Hoku was prompted to not answer questions outside the hawaii.edu domain or the context provided. However, earlier in development, it was possible to prompt Hoku to answer questions unrelated to ITS.

Prompt:

Response:

What was done to prevent this

Managing data access

Currently, Hoku’s database consists of only publicly available information. However, if Hoku’s knowledge is expanded to private/protected information, data privacy would be a great concern. Prompt injections could possibly lead to a leak of private/sensitive data.

What did we do to prepare for this?

Currently, for demonstration purposes, password login is enabled for the ITS admins. However, considerations were made to allow for authentication with Google. This will allow admins to login via the UH login portal that supports duo factor authentication. Making duo factor authentication mandatory for all ITS admins on the site is a good way to prevent unauthorized access.

In the Future: Self Hosted LLM

Even if prompt injections were made impossible and unauthorized data access is perfectly restricted. We would still have to send sensitive data over the internet to Open AI’s GPT models for response generation. Data could possibly be intercepted, or this data could be used by OpenAI to train next generation models (it would be very important to view OpenAI’s data usage policy).

This problem could be solved by a self-hosted LLM that Manoa would run on site. This does have its own problems, cost being one of them. But at least we would know exactly how our data is managed.

For a full list of scraped websites, Click Here

Deployment

Our application was deployed using Digital Ocean. The link to our application is here.

Development History

The development process conformed to Issue Driven Project Management practices. In quick summary:

Our development history is displayed in the sections below.

Milestone 1:

The goal of Milestone 1 is to improve the UI for the FAQ and report pages along with the chatbox. In addition, we plan to continue programming the chatbot AI by expanding the databases to accomplish the bonus challenges for HACC and focus on prompt injections. We also finalized our README.md and updated our GitHub Pages.

Milestone 1 was managed using GitHub Project Board M1:

Milestone 2:

The goal of Milestone 2 is to focus on creating the Forums page and adding a ‘Add Forum’ button. We are also looking into allowing users to comment on other users’ posts. Although we got majority of the features working, some of our code requires clean up.

Milestone 2 was managed using GitHub Project Board M2:

Milestone 3:

The goal of Milestone 3 is to finalize the Forums page by cleaning up the UI and fixing any issues in the backend. Users are able to add a forum or edit and delete their own forum posts. Admins can even delete any forum posts if it is inappropriate. We also finalized our GitHub Pages.

Milestone 3 was managed using GitHub Project Board M3:

Team

This application is designed, implemented, and maintained by team DarkMode which consists of Kaylee Agorilla, Malisa Lo, and Micah Tilton.