From 3d9c3cfd9b5185743f417b22fcfdd809c0286441 Mon Sep 17 00:00:00 2001 From: Phil Date: Sat, 25 Jan 2025 22:36:18 +0000 Subject: [PATCH] Updated README.md file and added log file and directory structure --- README.md | 59 ++++++++++++++++++++++++++++--------------- geolite2-db-update.sh | 35 +++++++++++++++++++------ log/download.log | 0 3 files changed, 65 insertions(+), 29 deletions(-) create mode 100644 log/download.log diff --git a/README.md b/README.md index c098fb1..8e18b6b 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ Scripts and configuration files for quickly setting up the echo lookup server created by mpolden - https://github.com/mpolden/echoip. -The main purpose of the project in this repository was to create config files for my setup which can be quickly deployed, in the even I needed to rebuild the service. +The main purpose of the project in this repository was to create config files for my setup which can be quickly deployed, in the event I needed to rebuild the service. This project provides an easy and fast way to set up EchoIP, a service that offers IP geolocation lookups, using the MaxMind GeoLite2 databases (City, ASN, and Country). The goal of this project is to streamline the deployment of EchoIP by automating the process of downloading, extracting, and organizing the required GeoLite2 data files, then running the service in a Docker container. @@ -23,13 +23,16 @@ Before setting up the project, ensure the following tools are installed: ## Setup Instructions -### 1. Download GeoLite2 Databases +### 1. **Clone the repository:** -To download the required databases (GeoLite2-City, GeoLite2-ASN, and GeoLite2-Country), you can run the provided Bash script. +``` +git clone https://git.ncltech.co.uk/phil/echoip-server-setup +cd echoip-server-setup +``` -#### Bash Script (`download_geoip.sh`) +### 2. **Downloading GeoLite2 Databases:** -The `download_geoip.sh` script will: +To download the required databases (GeoLite2-City, GeoLite2-ASN, and GeoLite2-Country), you can run geolite2-db-update.sh script. This will: - Download the required GeoLite2 databases from MaxMind. - Extract the `.tar.gz` files. @@ -37,23 +40,26 @@ The `download_geoip.sh` script will: #### Configuration +- Open the `geolite2-db-update.sh` script and replace `YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY` with your actual MaxMind account credentials. + - Set your MaxMind account ID and license key in the `AUTH` variable: ```bash - AUTH="YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY" +AUTH="YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY" ``` -The downloaded and extracted database files will be placed in the geolite2/db directory structure. + +- Change the `/path/to/default/main/data/folder` to the location where the project was downloaded or will be run from. The downloaded and extracted database files will be placed in the geolite2/db directory structure. Run the script: + ``` -chmod +x download_geoip.sh -./download_geoip.sh +chmod +x geolite2-db-update.sh +./geolite2-db-update.sh ``` -### 2. Docker Setup for EchoIP +### 3. **Docker Setup for EchoIP:** The Docker container uses the mpolden/echoip:latest image, which serves IP geolocation data based on the GeoLite2 databases. The docker-compose.yml file is provided to set up the service. -Docker Compose Configuration (docker-compose.yml) This file defines the EchoIP service and mounts the database files for the container to access. It also binds the service to port 8085. Configuration @@ -71,10 +77,9 @@ Start the Docker container: docker-compose up -d ``` -### 3. Apache Setup +### 4. **Apache Setup:** To proxy requests to the EchoIP service, you'll need to configure your Apache HTTP server. -Apache Configuration (apache_config.conf) Add the following configuration to your Apache server's configuration file (httpd.conf or a dedicated virtual host file). This configures Apache to forward requests to the EchoIP service running on localhost:8085 and supports the X-Forwarded-For header. @@ -96,20 +101,31 @@ Add the following configuration to your Apache server's configuration file (http CustomLog ${APACHE_LOG_DIR}/echoip-access.log combined ``` + Replace FULLY.QUALIFIED.DOMAIN.NAME with your server's domain name. -Restart Apache: +- Enable the necessary Apache modules: + +``` +sudo a2enmod proxy +sudo a2enmod proxy_http +sudo a2enmod headers +sudo a2enmod rewrite +sudo a2enmod remoteip +``` + +- Restart Apache: ``` sudo systemctl restart apache2 ``` -Accessing the Service +### 5. **Accessing the Service:** After setting up Docker and Apache: - The EchoIP service will be accessible through Apache on the domain you configured. - You can query geolocation data by making requests to the configured server. +The EchoIP service will be accessible through Apache on the domain you configured. +You can query geolocation data by making requests to the configured server. For example, a request to http://FULLY.QUALIFIED.DOMAIN.NAME/ip/8.8.8.8 will return geolocation information for the IP address 8.8.8.8. @@ -120,6 +136,7 @@ File Structure: ├── apache_config.conf # Apache virtual host configuration ├── download_geoip.sh # Bash script for downloading and extracting GeoLite2 databases ├── docker-compose.yml # Docker Compose configuration for EchoIP service +├── log/ # Log storage for the geolite2-db-update.sh script └── geolite2/ └── db/ ├── city/ # GeoLite2-City database @@ -127,11 +144,11 @@ File Structure: └── country/ # GeoLite2-Country database ``` -Troubleshooting +### 6. **Troubleshooting:** -Ensure that your MaxMind credentials are correct in the download_geoip.sh script. -Check the Apache logs (/var/log/apache2/echoip-error.log) for any issues related to the proxy configuration. -If Docker fails to start the EchoIP container, check the Docker logs with docker logs echoip. +* Ensure that your MaxMind credentials are correct in the download_geoip.sh script. Every time the geolite2-db-update.sh script is ran it will log to the log/download.log file. +* Check the Apache logs (/var/log/apache2/echoip-error.log) for any issues related to the proxy configuration. +* If Docker fails to start the EchoIP container, check the Docker logs with docker logs echoip. diff --git a/geolite2-db-update.sh b/geolite2-db-update.sh index f69d76b..efd04d7 100644 --- a/geolite2-db-update.sh +++ b/geolite2-db-update.sh @@ -1,35 +1,54 @@ #!/bin/bash + +# Define a default path for the main data folder +DEFAULT_MAIN_DATA_DIR="/path/to/default/main/data/folder" + +# Define your MaxMind account access for GeoLite2 databases +AUTH="YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY" + +# Script +# Use the provided argument or fall back to the default +MAIN_DATA_DIR="${1:-$DEFAULT_MAIN_DATA_DIR}" + +# Check if the specified main data folder exists +if [ ! -d "$MAIN_DATA_DIR" ]; then + echo "Error: The specified main data folder does not exist: $MAIN_DATA_DIR" + exit 1 +fi + # Define constants BASE_URL="https://download.maxmind.com/geoip/databases" -AUTH="YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY" -OUTPUT_DIR="$(dirname "$0")/geolite2/db" +OUTPUT_DIR="$MAIN_DATA_DIR/geolite2/db" # Geolite folder +LOG_DIR="$MAIN_DATA_DIR/log" # Log folder +LOG_FILE="$LOG_DIR/download.log" # Log file path CITY_FILE="GeoLite2-City" ASN_FILE="GeoLite2-ASN" COUNTRY_FILE="GeoLite2-Country" -# Create directory structure if it doesn't exist +# Create necessary directory structures if they don't exist mkdir -p "$OUTPUT_DIR/city" mkdir -p "$OUTPUT_DIR/asn" mkdir -p "$OUTPUT_DIR/country" +mkdir -p "$LOG_DIR" # Download, extract, and move the files process_file() { local file_name=$1 local target_dir=$2 - echo "Processing $file_name..." + echo "Processing $file_name..." | tee -a "$LOG_FILE" # Download the .tar.gz file to the output directory - curl -o "$OUTPUT_DIR/$file_name.tar.gz" -J -L -u "$AUTH" "$BASE_URL/$file_name/download?suffix=tar.gz" + curl -o "$OUTPUT_DIR/$file_name.tar.gz" -J -L -u "$AUTH" "$BASE_URL/$file_name/download?suffix=tar.gz" 2>> "$LOG_FILE" # Extract the .tar.gz file into the target directory - tar -xzf "$OUTPUT_DIR/$file_name.tar.gz" --wildcards --strip-components=1 -C "$target_dir" "*/$file_name.mmdb" + tar -xzf "$OUTPUT_DIR/$file_name.tar.gz" --wildcards --strip-components=1 -C "$target_dir" "*/$file_name.mmdb" 2>> "$LOG_FILE" # Clean up the downloaded tar.gz file rm -f "$OUTPUT_DIR/$file_name.tar.gz" - echo "$file_name processed and placed in $target_dir" + echo "$file_name processed and placed in $target_dir" | tee -a "$LOG_FILE" } # Process each database @@ -37,4 +56,4 @@ process_file "$CITY_FILE" "$OUTPUT_DIR/city" process_file "$ASN_FILE" "$OUTPUT_DIR/asn" process_file "$COUNTRY_FILE" "$OUTPUT_DIR/country" -echo "All files processed successfully." +echo "All files processed successfully." | tee -a "$LOG_FILE" diff --git a/log/download.log b/log/download.log new file mode 100644 index 0000000..e69de29