Virtual TA π
Last Updated: 2025-05-31
Overview
Virtual-TA is a Teaching Assistant Discourse responder designed for the IITM BS TDS (Tools in Data Science) course. This project aims to automate responses to common student queries and assist in managing course-related discussions.
Project Status: Active Development
β¨ Features
- Automated Responses: Provides quick answers to frequently asked questions.
- Discourse Integration: Seamlessly works with discourse platforms.
- Course-Specific: Tailored for the IITM BS TDS course curriculum and common queries.
- Extensible: Designed to be easily updated with new Q&A pairs and functionalities.
π οΈ Tech Stack
- Language: Python
- Deployment: Docker (Dockerfile and docker-compose.yml provided)
π Project Structure
PythonicVarun/Virtual-TA/
βββ .github/
β βββ ISSUE_TEMPLATE/
β β βββ bug_report.md
β β βββ feature_request.md
β βββ PULL_REQUEST_TEMPLATE.md
βββ app/ # Main application code
βββ scripts/ # Utility and automation scripts
βββ tests/ # Test suites
βββ .env.example # Example environment variables
βββ .gitignore # Specifies intentionally untracked files that Git should ignore
βββ CODE_OF_CONDUCT.md # Community guidelines
βββ CONTRIBUTING.md # Guidelines for contributors
βββ Dockerfile # Defines the Docker image
βββ LICENSE # Project license (MIT License)
βββ README.md # Project overview and instructions
βββ docker-compose.yml # Docker Compose configuration
βββ requirements.txt # Python dependencies
π Getting Started
Prerequisites
- Python 3.x
- Docker (Optional, for containerized deployment)
Installation
-
Clone the repository:
git clone https://github.com/PythonicVarun/Virtual-TA.git
cd Virtual-TA
-
Create and activate a virtual environment (recommended):
python -m venv venv
source venv/bin/activate # On Windows use `venv\Scripts\activate`
-
Install dependencies:
pip install -r requirements.txt
-
Set up environment variables:
Rename .env.example to .env and fill in the necessary configuration details.
cp .env.example .env
# Open .env and update variables
Running the Application
-
Directly with Python:
# (Ensure your virtual environment is activated and .env is configured)
# Runs at localhost:8000
uvicorn app.main:app --reload
-
With Docker:
- Build the Docker image:
docker build -t virtual-ta .
- Run using Docker Compose:
Or run the image directly:
docker run -d --env-file .env virtual-ta
βοΈ Scripts
This directory contains various utility and automation scripts for data preparation, processing, and automation:
- Purpose: Downloads all Discourse threads in a specific category between two dates, including all posts in each thread (with pagination support). Supports authentication via API key or cookies.
- Usage:
python scripts/get_discourse_threads.py \
--base-url "https://discourse.onlinedegree.iitm.ac.in" \
--category-path "courses/tds-kb/34" \
--start-date "2025-01-01" \
--end-date "2025-04-14" \
--output-dir "data/raw_discourse_threads" \
[--api-key KEY --api-username USER] \
[--cookies "name=value; name2=value2"]
- Purpose: Converts Discourse thread JSON files into plain-text Markdown summaries. Can process a single file or all JSON files in a directory.
- Usage:
python scripts/jsonpost2text.py input_path [--output OUTPUT_DIR]
# Example:
python scripts/jsonpost2text.py thread.json
python scripts/jsonpost2text.py data/raw_discourse_threads --output data/discourse_posts
- Purpose: Downloads a GitHub repository (by branch or commit) as a ZIP, extracts it, and keeps only
.md files, deleting everything else.
- Usage:
python scripts/get_course_content.py [--branch BRANCH | --commit SHA] <repo_url> [--output OUTPUT_DIR]
# Example:
python scripts/get_course_content.py --branch tds-2025-01 --output data/course_content https://github.com/sanand0/tools-in-data-science-public
- Purpose: Builds a FAISS vector database from text, Markdown, or HTML files (e.g., course content or discourse posts). Chunks and embeds text using OpenAI API, and saves the index and metadata for later retrieval.
- Usage:
Edit configuration variables if needed, then run:
python scripts/create_vector_db.py
By default, processes content in the data/ directory and saves output to model/.
For more details about each script, see the script source files in the scripts/ directory.
π License
This project is licensed under the MIT License - see the LICENSE file for details.
π€ Contributing
Contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
Please read our Contributing Guidelines for details on our code of conduct, and the process for submitting pull requests to us.
Code of Conduct
We have a Code of Conduct that we expect all contributors and community members to adhere to. Please read it to understand the expectations.
π§ Contact
PythonicVarun (hello@pythonicvarun.me)