Managing CVE Data Locally with CVE Database Manager

Introduction

In the cybersecurity landscape, keeping track of vulnerabilities is crucial for maintaining secure systems. The Common Vulnerabilities and Exposures (CVE) list is a comprehensive catalog of such vulnerabilities. However, using public APIs like the National Vulnerability Database (NVD) can be limiting due to rate limits and other complexities. To address these issues, I created the CVE Database Manager, a repository that allows you to manage a local CVE database and serve the data via a FastAPI application.

Repository URL:CVE Database Manager

Repository Overview

The CVE Database Manager provides scripts and configurations to set up a PostgreSQL database, populate it with CVE data from the official CVE list, and serve this data through a FastAPI-based API. This setup is particularly useful for air-gapped environments where external API access is restricted.

Why Use a Local CVE Database?

• Avoid Rate Limits: Bypass the rate limits imposed by public APIs.

• Speed: Faster access to CVE data.

• Customization: Ability to customize and extend the database as needed.

• Security: Suitable for air-gapped or highly secure environments.

Setup Guide

Note: For a quick reference on setting up and running the CVE Database Manager, including PostgreSQL installation, database creation, and application setup, please refer to the Setup Cheat Sheet.

Prerequisites

Before setting up the CVE Database Manager, ensure you have the following:

• PostgreSQL (version 12 or later)

• Python 3.x (preferably 3.8 or later)

Step-by-Step Installation

1. Install PostgreSQL

   For Debian/Ubuntu:

    sudo apt update
    sudo apt install postgresql postgresql-contrib
   

   For CentOS/RHEL:

    sudo yum install postgresql-server postgresql-contrib
    sudo postgresql-setup initdb
    sudo systemctl start postgresql
    sudo systemctl enable postgresql
   

   For more detailed instructions, refer to the Setup Cheat Sheet.

2. Set Up PostgreSQL Database

   Switch to the PostgreSQL user and create a new user and database:

    sudo -i -u postgres
    psql
    CREATE USER your_username WITH PASSWORD 'your_password';
    CREATE DATABASE your_database;
    GRANT ALL PRIVILEGES ON DATABASE your_database TO your_username;
    \q
    exit
   

3. Clone the Repository

    git clone https://github.com/iam-niranjan/cve-database-manager.git
    cd cve-database-manager
   

4. Install Dependencies

    pip install -r requirements.txt
   

5. Clone the CVE Data Repository

    git clone https://github.com/CVEProject/cvelistV5.git
   

6. Create Database Schema

   Refer to the db_schema.sql script to create the necessary tables and views in your PostgreSQL database. This script should be executed within the PostgreSQL shell.

7. Update the Database

   To populate and update the database with the latest CVE data:

   • Initial Data Population:

     Run the update_cve_db.py script provided in the repository to initially populate the database with CVE data from the local repository.

   • Future Updates:

     Regularly run the update_cve_db.py script to fetch and update the database with any new or modified CVE data from the local repository. Refer to the update_cve_db.py script in the repository for detailed instructions.

8. Running the FastAPI Application

   To serve the CVE data through an API (api.py)using FastAPI:

   • Start the FastAPI application:

       uvicorn api:app --host 0.0.0.0 --port 8000
     

   • Access the API Documentation:

     Since the application uses FastAPI, it automatically includes detailed Swagger documentation. This can be accessed locally at:

     • Swagger UI: http://localhost:8000/docs

     • ReDoc: http://localhost:8000/redoc

These interfaces provide a comprehensive overview of the API endpoints, allowing you to interact with the API directly from your browser and explore the available operations and data structures.

Usage

Once the database is populated and the FastAPI server is running, you can access the CVE data through the API endpoints. For example, to get details of a specific CVE:

curl -H "X-API-Key: <your_api_key>" http://localhost:8000/cve/CVE-2023-0001

Note: Use the API key specified in your configuration or generated for security purposes.

Conclusion

Managing a local CVE database offers numerous benefits, from avoiding API rate limits to having faster and more reliable access to vulnerability data. With the CVE Database Manager, you can easily set up and maintain your own local CVE database, making it a valuable tool for any security-conscious organization. For more details and to get started, visit the CVE Database Manager repository.