Introduction
In the cybersecurity landscape, keeping track of vulnerabilities is crucial for maintaining secure systems. The Common Vulnerabilities and Exposures (CVE) list is a comprehensive catalog of such vulnerabilities. However, using public APIs like the National Vulnerability Database (NVD) can be limiting due to rate limits and other complexities. To address these issues, I created the CVE Database Manager, a repository that allows you to manage a local CVE database and serve the data via a FastAPI application.
Repository URL:CVE Database Manager
Repository Overview
The CVE Database Manager provides scripts and configurations to set up a PostgreSQL database, populate it with CVE data from the official CVE list, and serve this data through a FastAPI-based API. This setup is particularly useful for air-gapped environments where external API access is restricted.
Why Use a Local CVE Database?
Avoid Rate Limits: Bypass the rate limits imposed by public APIs.
Speed: Faster access to CVE data.
Customization: Ability to customize and extend the database as needed.
Security: Suitable for air-gapped or highly secure environments.
Setup Guide
Note: For a quick reference on setting up and running the CVE Database Manager, including PostgreSQL installation, database creation, and application setup, please refer to the Setup Cheat Sheet.
Prerequisites
Before setting up the CVE Database Manager, ensure you have the following:
Step-by-Step Installation
Install PostgreSQL
For Debian/Ubuntu:
sudo apt update
sudo apt install postgresql postgresql-contrib
For CentOS/RHEL:
sudo yum install postgresql-server postgresql-contrib
sudo postgresql-setup initdb
sudo systemctl start postgresql
sudo systemctl enable postgresql
For more detailed instructions, refer to the Setup Cheat Sheet.
Set Up PostgreSQL Database
Switch to the PostgreSQL user and create a new user and database:
sudo -i -u postgres
psql
CREATE USER your_username WITH PASSWORD 'your_password';
CREATE DATABASE your_database;
GRANT ALL PRIVILEGES ON DATABASE your_database TO your_username;
\q
exit
Clone the Repository
git clone https://github.com/iam-niranjan/cve-database-manager.git
cd cve-database-manager
Install Dependencies
pip install -r requirements.txt
Clone the CVE Data Repository
git clone https://github.com/CVEProject/cvelistV5.git
Create Database Schema
Refer to the db_schema.sql script to create the necessary tables and views in your PostgreSQL database. This script should be executed within the PostgreSQL shell.
Update the Database
To populate and update the database with the latest CVE data:
Initial Data Population:
Run the update_cve_db.py script provided in the repository to initially populate the database with CVE data from the local repository.
Future Updates:
Regularly run the update_cve_db.py script to fetch and update the database with any new or modified CVE data from the local repository. Refer to the update_cve_db.py script in the repository for detailed instructions.
Running the FastAPI Application
To serve the CVE data through an API (api.py)using FastAPI:
Start the FastAPI application:
uvicorn api:app --host 0.0.0.0 --port 8000
Access the API Documentation:
Since the application uses FastAPI, it automatically includes detailed Swagger documentation. This can be accessed locally at:
These interfaces provide a comprehensive overview of the API endpoints, allowing you to interact with the API directly from your browser and explore the available operations and data structures.
Usage
Once the database is populated and the FastAPI server is running, you can access the CVE data through the API endpoints. For example, to get details of a specific CVE:
curl -H "X-API-Key: <your_api_key>" http://localhost:8000/cve/CVE-2023-0001
Note: Use the API key specified in your configuration or generated for security purposes.
Conclusion
Managing a local CVE database offers numerous benefits, from avoiding API rate limits to having faster and more reliable access to vulnerability data. With the CVE Database Manager, you can easily set up and maintain your own local CVE database, making it a valuable tool for any security-conscious organization. For more details and to get started, visit the CVE Database Manager repository.
