GitHubSentinel

GitHub Sentinel

English | Chinese

Table of contents

• GitHub Sentinel
• Main functions
• Product screenshot
• Start quickly
  • 1. Installation dependencies
  • 2. Configure the application
  • 3. How to run
    • A. Run as a command line tool
    • B. Run as a background service
    • C. Run as a Gradio server
• Ollama installation and service release
• Unit Testing
  • Unit testing and validation script validate_tests.sh
    • use
    • Function
• Build and Verify with Docker
  • 1. Dockerfile
    • use
    • Key Steps
  • 2. build_image.sh
    • use
    • Function
• contribute
• license
• connect

GitHub Sentinel is AI Agent designed for intelligent information retrieval and high-value content mining specially designed for the era of big models (LLMs). It is aimed at users who need high frequency and large amounts of information acquisition, especially open source enthusiasts, individual developers and investors.

Main functions

• Subscription Management : Easily manage and track the GitHub repository you follow.
• Update Retrieval : Automatically retrieve and summarize the latest updates of subscription repositories, including submission records, questions, and pull requests.
• Notification system : Notify subscribers of the latest progress of the project in real time through emails and other means.
• Report generation : Generate detailed project progress reports based on the retrieved updates, supporting multiple formats and templates to meet different needs.
• Multi-model support : Combining OpenAI and Ollama models, natural language project reports are generated to provide smarter and more accurate information services.
• Timing tasks : Supports execution of timing tasks in a daemon way to ensure timely information updates and obtains.
• Graphical interface : Based on Gradio, it realizes a simple and easy-to-use GUI operation mode, lowering the threshold for use.
• Containerization : The project supports Docker construction and containerized deployment, which facilitates rapid deployment and operation in different environments.
• Continuous integration : Complete unit testing is implemented to facilitate further configuration of production-level CI/CD processes to ensure project stability and high-quality delivery.

GitHub Sentinel not only helps users automatically track and analyze the latest updates of GitHub 开源项目, but also quickly expand to other information channels, such as the popular topics of Hacker News , providing more comprehensive information mining and analysis capabilities.

Product screenshot

GitHub project progress tracking and summary

Hacker News Hot Tech Topic Mining

Start quickly

1. Installation dependencies

First, install the required dependencies:

pip install -r requirements.txt

2. Configure the application

Edit the config.json file to set your GitHub Token, Email settings (taking Tencent WeChat Mailbox as an example), subscription files, update settings, big model service configuration (supports OpenAI GPT API and Ollama's private big model service), and automatically retrieve and generate reports (GitHub project progress, Hacker News hot topics and cutting-edge technology trends):

{
    "github" : {
        "token" : " your_github_token " ,
        "subscriptions_file" : " subscriptions.json " ,
        "progress_frequency_days" : 1 ,
        "progress_execution_time" : " 08:00 "
    },
    "email" :  {
        "smtp_server" : " smtp.exmail.qq.com " ,
        "smtp_port" : 465 ,
        "from" : " [email protected] " ,
        "password" : " your_email_password " ,
        "to" : " [email protected] "
    },
    "llm" : {
        "model_type" : " ollama " ,
        "openai_model_name" : " gpt-4o-mini " ,
        "ollama_model_name" : " llama3 " ,
        "ollama_api_url" : " http://localhost:11434/api/chat "
    },
    "report_types" : [
        " github " ,
        " hacker_news_hours_topic " ,
        " hacker_news_daily_report "
    ],
    "slack" : {
        "webhook_url" : " your_slack_webhook_url "
    }
}

For security reasons: Both GitHub Token and Email Password settings support configuration using environment variables to avoid explicitly configuring important information, as shown below:

 # Github
export GITHUB_TOKEN= " github_pat_xxx "
# Email
export EMAIL_PASSWORD= " password "

3. How to run

GitHub Sentinel supports the following three ways of running:

A. Run as a command line tool

You can run the app interactively from the command line:

python src/command_tool.py

In this mode, you can manually enter commands to manage subscriptions, retrieve updates, and generate reports.

B. Run as a background service

To run the app as a background service (daemon), it will be updated automatically based on the relevant configuration.

You can start, query status, close and restart directly using daemon management script daemon_control.sh:

1. Start the service:

   $ ./daemon_control.sh start
   Starting DaemonProcess...
   DaemonProcess started.

   • This will start [./src/daemon_process.py], generate reports regularly according to the update frequency and time points set in config.json , and send emails.
   • This service log will be saved to the logs/DaemonProcess.log file. At the same time, the historical cumulative logs will also be synchronously appended to logs/app.log log file.

2. Query service status:

   $ ./daemon_control.sh status
   DaemonProcess is running.

3. Close the service:

   $ ./daemon_control.sh stop
   Stopping DaemonProcess...
   DaemonProcess stopped.

4. Restart the service:

   $ ./daemon_control.sh restart
   Stopping DaemonProcess...
   DaemonProcess stopped.
   Starting DaemonProcess...
   DaemonProcess started.

C. Run as a Gradio server

To run an application using the Gradio interface, allow users to interact with the tool through the web interface:

python src/gradio_server.py

• This will start a web server on your machine, allowing you to manage subscriptions and generate reports through a user-friendly interface.
• By default, the Gradio server will be accessible at http://localhost:7860 , but you can share it publicly if you want.

Ollama installation and service release

Ollama is a private big model management tool that supports local and containerized deployment, command-line interaction, and REST API calls.

For detailed instructions on the release of Ollama installation deployment and privatization mockup service, please refer to the release of Ollama installation deployment and service.

Ollama brief official installation

To call the private mockup service using Ollama in GitHub Sentinel, follow these steps to install and configure:

1. Install Ollama : Please download and install the Ollama service according to Ollama's official documentation. Ollama supports a variety of operating systems, including Linux, Windows, and macOS.

2. Start the Ollama service : After the installation is complete, start the Ollama service with the following command:

   ollama serve

   By default, the Ollama API will run at http://localhost:11434 .

3. Configuring Ollama Use in GitHub Sentinel : In the config.json file, configure the Ollama API related information:

   {
       "llm" : {
           "model_type" : " ollama " ,
           "ollama_model_name" : " llama3 " ,
           "ollama_api_url" : " http://localhost:11434/api/chat "
       }
   }

4. Verify the configuration : Start GitHub Sentinel with the following command and generate a report to verify that the Ollama configuration is correct:

   python src/command_tool.py

   If configured correctly, you will be able to generate reports through the Ollama model.

Unit Testing

To ensure the quality and reliability of the code, GitHub Sentinel uses the unittest module for unit testing. For detailed descriptions of unittest and its related tools such as @patch and MagicMock , please refer to the unit test details.

Unit testing and validation script validate_tests.sh

use

validate_tests.sh is a Shell script that runs unit tests and validates the results. It is executed during the Docker image building process to ensure the correctness and stability of the code.

Function

• The script runs all unit tests and outputs the results to test_results.txt file.
• If the test fails, the script outputs the test results and causes the Docker build to fail.
• If all tests pass, the script continues to build the process.

Build and Verify with Docker

To facilitate the construction and deployment of GitHub Sentinel projects in a variety of environments, we provide Docker support. This support includes the following files and features:

1. Dockerfile

use

Dockerfile is a configuration file used to define how to build a Docker image. It describes the steps to build the image, including installing dependencies, copying project files, running unit tests, etc.

Key Steps

• Use python:3.10-slim as the base image and set the working directory to /app .
• Copy the project's requirements.txt file and install the Python dependencies.
• Copy all files of the project to the container and give validate_tests.sh script execution permission.
• Execute the validate_tests.sh script during the build process to ensure that all unit tests pass. If the test fails, the build process will abort.
• After the build is successful, src/main.py will be run by default as the entry point of the container.

2. build_image.sh

use

build_image.sh is a shell script for automatically building Docker images. It takes the branch name from the current Git branch and uses it as a label for the Docker image, making it easier to generate different Docker images on different branches.

Function

• Gets the current Git branch name and uses it as a label for the Docker image.
• Use the docker build command to build the Docker image and use the current Git branch name as the label.

Example of usage

chmod +x build_image.sh
./build_image.sh

Through these scripts and configuration files, it is ensured that in different development branches, the Docker images built are based on unit-tested code, thereby improving code quality and deployment reliability.

contribute

Contribution is what makes the open source community amazing to learn, inspire and create. Thank you very much for any contribution you have made. If you have any suggestions or feature requests, please start a topic to discuss what you want to change.

license

The project is licensed under the terms of the Apache-2.0 license. See the LICENSE file for details.

connect

Django Peng - [email protected]

Project link: https://github.com/DjangoPeng/GitHubSentinel