mirror of
https://github.com/C24Be/AS_Network_List.git
synced 2026-03-26 11:20:59 +03:00
Compare commits
1 Commits
main
...
48afd374fc
| Author | SHA1 | Date | |
|---|---|---|---|
|
48afd374fc |
4
.github/workflows/update_blacklists.yml
vendored
4
.github/workflows/update_blacklists.yml
vendored
@@ -32,8 +32,6 @@ jobs:
|
||||
- run: ./blacklists_updater_txt.sh
|
||||
- run: ./blacklists_updater_nginx.sh
|
||||
- run: ./blacklists_updater_iptables.sh
|
||||
- run: ./blacklists_updater_nftables.sh
|
||||
- run: ./blacklists_updater_routes.sh
|
||||
- uses: ./.github/actions/gitPush
|
||||
env:
|
||||
PUSH_FILES: blacklists/ blacklists_nginx/ blacklists_iptables/ blacklists_nftables/
|
||||
PUSH_FILES: blacklists/ blacklists_nginx/ blacklists_iptables/
|
||||
|
||||
33
.github/workflows/update_nftables.yml
vendored
33
.github/workflows/update_nftables.yml
vendored
@@ -1,33 +0,0 @@
|
||||
name: Update nftables Blacklists
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '30 2 * * *' # Daily at 02:30 UTC
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
update-nftables:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.x'
|
||||
|
||||
- name: Generate nftables blacklists
|
||||
run: |
|
||||
chmod +x blacklists_updater_nftables.sh
|
||||
./blacklists_updater_nftables.sh
|
||||
|
||||
- name: Commit and push if changed
|
||||
run: |
|
||||
git config --local user.email "github-actions[bot]@users.noreply.github.com"
|
||||
git config --local user.name "github-actions[bot]"
|
||||
git add blacklists_nftables/
|
||||
git diff --quiet && git diff --staged --quiet || \
|
||||
(git commit -m "Auto-update nftables blacklists [skip ci]" && git push)
|
||||
|
||||
73
README.md
73
README.md
@@ -2,11 +2,6 @@
|
||||
|
||||
### Blacklists are updated daily!
|
||||
|
||||
> [!IMPORTANT]
|
||||
> A very important feature has been added: dedicated lists of VK Cloud / VK networks that can be used to block **OUTGOING** traffic from your server (iptables/ipset and nftables formats are available).
|
||||
> This can help reduce the risk of Messenger MAX being used to compromise your VPN server.
|
||||
> The best security option is to avoid installing Messenger MAX at all on a phone where VPN access is configured.
|
||||
|
||||
This repository contains Python scripts that allow you to retrieve network lists based on either an Autonomous System (AS) name or a Network name. Also you can download and parse the whole RIPE database to get information about Networks for the further analysis.
|
||||
|
||||
**Ready-to-use blacklists in multiple formats:**
|
||||
@@ -14,8 +9,6 @@ This repository contains Python scripts that allow you to retrieve network lists
|
||||
- [Text blacklists in `blacklists/`](https://github.com/C24Be/AS_Network_List/tree/main/blacklists) - Plain text format with IPv4/IPv6 separation
|
||||
- [Nginx configurations in `blacklists_nginx/`](https://github.com/C24Be/AS_Network_List/tree/main/blacklists_nginx) - Ready to include in your nginx config
|
||||
- [IPTables/IPSet files in `blacklists_iptables/`](https://github.com/C24Be/AS_Network_List/tree/main/blacklists_iptables) - Optimized for iptables with ipset
|
||||
- [nftables files in `blacklists_nftables/`](https://github.com/C24Be/AS_Network_List/tree/main/blacklists_nftables) - Ready-to-load sets and rules for nftables
|
||||
- [Linux route files in `blacklists_route/`](https://github.com/C24Be/AS_Network_List/tree/main/blacklists_route) - VK route blackholes to loopback (IPv4/IPv6)
|
||||
- [Other network and ASN lists in `auto/`](https://github.com/C24Be/AS_Network_List/tree/main/auto) - Comprehensive Russian network data
|
||||
|
||||
## Files and features
|
||||
@@ -27,16 +20,12 @@ This repository contains Python scripts that allow you to retrieve network lists
|
||||
- `get_info_from_ripe.py`: Retrieves information about Russian AS numbers and Networks from RIPE database for the further analysis.
|
||||
- `get_description.py`: Retrieves network names, AS names and organisation names from RIPE. Updates the lists in the folder `auto/`.
|
||||
- `parse_ripe_db.py`: Parses the whole RIPE database to get information about Networks for the further analysis.
|
||||
- `generate_nft_blacklist.py`: Takes text blacklist on the input and generates nftables config with sets.
|
||||
- `check_nft_blacklist.py`: Checks IPv4/IPv6 address against generated nftables config.
|
||||
|
||||
### Blacklist Generators
|
||||
|
||||
- `blacklists_updater_txt.sh`: Generates text-based blacklists with IPv4/IPv6 separation
|
||||
- `blacklists_updater_nginx.sh`: Generates nginx configuration files with deny directives
|
||||
- `blacklists_updater_iptables.sh`: Generates ipset configuration files for iptables/ip6tables
|
||||
- `blacklists_updater_nftables.sh`: Generates nftables blacklist files (mixed/v4/v6 and VK-specific)
|
||||
- `blacklists_updater_routes.sh`: Generates Linux route files to send VK networks to loopback (`127.0.0.1` / `::1`)
|
||||
|
||||
### Generated Blacklists
|
||||
|
||||
@@ -61,22 +50,6 @@ This repository contains Python scripts that allow you to retrieve network lists
|
||||
- `blacklist-v6.ipset`: IPSet configuration for IPv6 only (**daily generated**)
|
||||
- `README.md`: Complete usage documentation for iptables integration
|
||||
|
||||
**nftables Format** (`blacklists_nftables/` folder):
|
||||
|
||||
* `blacklist.nft`: nftables configuration for mixed IPv4/IPv6 (**daily generated**)
|
||||
* `blacklist-v4.nft`: nftables configuration for IPv4 only (**daily generated**)
|
||||
* `blacklist-v6.nft`: nftables configuration for IPv6 only (**daily generated**)
|
||||
* `blacklist-vk.nft`: nftables configuration for VK-only networks (**daily generated**)
|
||||
* `blacklist-vk-v4.nft`: nftables configuration for VK-only IPv4 networks (**daily generated**)
|
||||
* `blacklist-vk-v6.nft`: nftables configuration for VK-only IPv6 networks (**daily generated**)
|
||||
* `README.md`: Complete usage documentation for nftables integration
|
||||
|
||||
**Linux Routes Format** (`blacklists_route/` folder):
|
||||
|
||||
* `blacklist-vk-v4.routes`: IPv4 routes for VK-only networks to `127.0.0.1` via `lo` (**daily generated**)
|
||||
* `blacklist-vk-v6.routes`: IPv6 routes for VK-only networks to `::1` via `lo` (**daily generated**)
|
||||
|
||||
|
||||
### Reference Lists
|
||||
|
||||
**Contributors are welcome!**
|
||||
@@ -117,38 +90,6 @@ iptables -I INPUT -m set --match-set blacklist-v4 src -j DROP
|
||||
ip6tables -I INPUT -m set --match-set blacklist-v6 src -j DROP
|
||||
```
|
||||
|
||||
**For nftables:**
|
||||
````bash
|
||||
# Download and load into nftables
|
||||
wget https://raw.githubusercontent.com/C24Be/AS_Network_List/main/blacklists_nftables/blacklist.nft
|
||||
sudo nft -f blacklist.nft
|
||||
|
||||
# Protect VM from incoming blacklist sources
|
||||
sudo nft add chain inet filter input '{ type filter hook input priority 0; policy accept; }'
|
||||
sudo nft add rule inet filter input ip saddr @blacklist_v4 counter reject
|
||||
sudo nft add rule inet filter input ip6 saddr @blacklist_v6 counter reject
|
||||
|
||||
# VK-only outbound blocking for VPN clients via NAT/FORWARD
|
||||
wget https://raw.githubusercontent.com/C24Be/AS_Network_List/main/blacklists_nftables/blacklist-vk.nft
|
||||
sudo nft -f blacklist-vk.nft
|
||||
sudo nft add chain inet filter forward '{ type filter hook forward priority 0; policy accept; }'
|
||||
sudo nft add rule inet filter forward iifname "<VPN_IFACE>" ip daddr @blacklist_v4 counter reject
|
||||
sudo nft add rule inet filter forward iifname "<VPN_IFACE>" ip6 daddr @blacklist_v6 counter reject
|
||||
|
||||
# View the loaded rules
|
||||
sudo nft list ruleset
|
||||
````
|
||||
|
||||
**For Linux Routes (VK loopback blackhole):**
|
||||
|
||||
```bash
|
||||
# Download and apply VK route files
|
||||
wget https://raw.githubusercontent.com/C24Be/AS_Network_List/main/blacklists_route/blacklist-vk-v4.routes
|
||||
wget https://raw.githubusercontent.com/C24Be/AS_Network_List/main/blacklists_route/blacklist-vk-v6.routes
|
||||
sudo sh blacklist-vk-v4.routes
|
||||
sudo sh blacklist-vk-v6.routes
|
||||
```
|
||||
|
||||
**For Custom Applications:**
|
||||
|
||||
```bash
|
||||
@@ -242,20 +183,6 @@ See the README files in each folder for detailed usage instructions.
|
||||
./network_list_from_netname.py --help
|
||||
```
|
||||
|
||||
### `generate_nft_blacklist.py`
|
||||
1. Generate nftables config from blacklist:
|
||||
|
||||
```bash
|
||||
./generate_nft_blacklist.py blacklists/blacklist.txt blacklist.nft
|
||||
```
|
||||
|
||||
### `check_nft_blacklist.py`
|
||||
1. Check IP address against generated config:
|
||||
|
||||
```bash
|
||||
./check_nft_blacklist.py blacklist.nft 77.37.166.239
|
||||
```
|
||||
|
||||
## Screenshots
|
||||
|
||||
<img width="320" alt="image" src="https://github.com/C24Be/AS_Network_List/assets/153936414/71bd0ed4-0e9b-42f0-8e91-01964ea9b8e1">
|
||||
|
||||
10908
auto/all-ru-asn.txt
10908
auto/all-ru-asn.txt
File diff suppressed because it is too large
Load Diff
9300
auto/all-ru-ipv4.txt
9300
auto/all-ru-ipv4.txt
File diff suppressed because it is too large
Load Diff
4365
auto/all-ru-ipv6.txt
4365
auto/all-ru-ipv6.txt
File diff suppressed because it is too large
Load Diff
185847
auto/all-ru.txt
185847
auto/all-ru.txt
File diff suppressed because it is too large
Load Diff
34787
auto/ripe-ru-ipv4.json
34787
auto/ripe-ru-ipv4.json
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
1165
blacklists/blacklist-v4.txt
Normal file
1165
blacklists/blacklist-v4.txt
Normal file
File diff suppressed because it is too large
Load Diff
25
blacklists/blacklist-v6.txt
Normal file
25
blacklists/blacklist-v6.txt
Normal file
@@ -0,0 +1,25 @@
|
||||
2a00:1148::/29
|
||||
2a00:1148::/32
|
||||
2a00:46e0:2::/48
|
||||
2a00:46e0::/32
|
||||
2a00:a300::/32
|
||||
2a00:b4c0::/32
|
||||
2a00:bdc0:8000::/34
|
||||
2a00:bdc0::/33
|
||||
2a00:bdc0:c000::/35
|
||||
2a00:bdc0:e002::/48
|
||||
2a00:bdc0:e003::/48
|
||||
2a00:bdc0:e004::/48
|
||||
2a00:bdc0:e005::/48
|
||||
2a00:bdc0:e007::/48
|
||||
2a00:bdc0:f000::/36
|
||||
2a00:bdc1::/32
|
||||
2a00:bdc2::/31
|
||||
2a00:bdc4::/30
|
||||
2a0c:a9c7:156::/48
|
||||
2a0c:a9c7:157::/48
|
||||
2a0c:a9c7:158::/48
|
||||
2a14:25c0::/32
|
||||
2a14:25c5::/32
|
||||
2a14:25c6::/32
|
||||
2a14:25c7::/32
|
||||
1190
blacklists/blacklist.txt
Normal file
1190
blacklists/blacklist.txt
Normal file
File diff suppressed because it is too large
Load Diff
2427
blacklists/blacklist_with_comments.txt
Normal file
2427
blacklists/blacklist_with_comments.txt
Normal file
File diff suppressed because it is too large
Load Diff
109
blacklists_iptables/README.md
Normal file
109
blacklists_iptables/README.md
Normal file
@@ -0,0 +1,109 @@
|
||||
# IPTables/IPSet Blacklist Configurations
|
||||
|
||||
Auto-generated ipset configuration files for blocking networks and IP addresses with iptables/ip6tables.
|
||||
|
||||
## Available Files
|
||||
|
||||
### IPv4 Only
|
||||
|
||||
- **`blacklist-v4.ipset`** - Contains only IPv4 networks (806 entries)
|
||||
|
||||
### IPv6 Only
|
||||
|
||||
- **`blacklist-v6.ipset`** - Contains only IPv6 networks (3 entries)
|
||||
|
||||
### Mixed IPv4/IPv6
|
||||
|
||||
- **`blacklist.ipset`** - Contains both IPv4 and IPv6 sets (809 total entries)
|
||||
|
||||
## Usage
|
||||
|
||||
### 1. Load the IPSet
|
||||
|
||||
```bash
|
||||
# For IPv4 only
|
||||
ipset restore < blacklist-v4.ipset
|
||||
|
||||
# For IPv6 only
|
||||
ipset restore < blacklist-v6.ipset
|
||||
|
||||
# For both IPv4 and IPv6 (loads both sets)
|
||||
ipset restore < blacklist.ipset
|
||||
```
|
||||
|
||||
### 2. Apply IPTables Rules
|
||||
|
||||
```bash
|
||||
# For IPv4
|
||||
iptables -I INPUT -m set --match-set blacklist-v4 src -j DROP
|
||||
iptables -I FORWARD -m set --match-set blacklist-v4 src -j DROP
|
||||
|
||||
# For IPv6
|
||||
ip6tables -I INPUT -m set --match-set blacklist-v6 src -j DROP
|
||||
ip6tables -I FORWARD -m set --match-set blacklist-v6 src -j DROP
|
||||
```
|
||||
|
||||
### 3. Persist Rules (Optional)
|
||||
|
||||
To make the rules persistent across reboots:
|
||||
|
||||
**On Debian/Ubuntu:**
|
||||
|
||||
```bash
|
||||
# Save iptables rules
|
||||
iptables-save > /etc/iptables/rules.v4
|
||||
ip6tables-save > /etc/iptables/rules.v6
|
||||
|
||||
# Save ipset
|
||||
ipset save > /etc/ipset.conf
|
||||
```
|
||||
|
||||
**On RHEL/CentOS:**
|
||||
|
||||
```bash
|
||||
# Save iptables rules
|
||||
service iptables save
|
||||
service ip6tables save
|
||||
|
||||
# Save ipset
|
||||
ipset save > /etc/sysconfig/ipset
|
||||
```
|
||||
|
||||
### 4. Update Existing Sets
|
||||
|
||||
To update the blacklist without restarting iptables:
|
||||
|
||||
```bash
|
||||
# Flush and reload
|
||||
ipset flush blacklist-v4
|
||||
ipset restore < blacklist-v4.ipset
|
||||
```
|
||||
|
||||
### 5. Remove Sets
|
||||
|
||||
```bash
|
||||
# Remove IPv4 set
|
||||
ipset flush blacklist-v4
|
||||
ipset destroy blacklist-v4
|
||||
|
||||
# Remove IPv6 set
|
||||
ipset flush blacklist-v6
|
||||
ipset destroy blacklist-v6
|
||||
```
|
||||
|
||||
## Performance Benefits
|
||||
|
||||
IPSet uses hash tables for O(1) lookup performance, making it ideal for large blacklists:
|
||||
|
||||
- Much faster than individual iptables rules
|
||||
- Minimal CPU overhead
|
||||
- Supports up to 65536 entries per set (configurable)
|
||||
- Kernel-level implementation for maximum efficiency
|
||||
|
||||
## Automatic Updates
|
||||
|
||||
These files are automatically regenerated when the blacklists are updated via the GitHub Actions workflow.
|
||||
|
||||
## Source
|
||||
|
||||
Generated from the blacklist files in the `blacklists/` directory.
|
||||
1183
blacklists_iptables/blacklist-v4.ipset
Normal file
1183
blacklists_iptables/blacklist-v4.ipset
Normal file
File diff suppressed because it is too large
Load Diff
43
blacklists_iptables/blacklist-v6.ipset
Normal file
43
blacklists_iptables/blacklist-v6.ipset
Normal file
@@ -0,0 +1,43 @@
|
||||
# IPSet blacklist configuration (IPv6 only)
|
||||
# Auto-generated from blacklist-v6.txt
|
||||
# Last updated: 2025-12-31 06:24:51 UTC
|
||||
#
|
||||
# Usage:
|
||||
# 1. Load the ipset:
|
||||
# ipset restore < blacklist-v6.ipset
|
||||
#
|
||||
# 2. Use with iptables/ip6tables:
|
||||
# iptables -I INPUT -m set --match-set blacklist-v6 src -m conntrack --ctstate NEW -j DROP
|
||||
# iptables -I FORWARD -m set --match-set blacklist-v6 src -m conntrack --ctstate NEW -j DROP
|
||||
#
|
||||
# 3. To flush/delete the set:
|
||||
# ipset flush blacklist-v6
|
||||
# ipset destroy blacklist-v6
|
||||
#
|
||||
|
||||
create blacklist-v6 hash:net family inet6 hashsize 1024 maxelem 50
|
||||
add blacklist-v6 2a00:1148::/29
|
||||
add blacklist-v6 2a00:1148::/32
|
||||
add blacklist-v6 2a00:46e0:2::/48
|
||||
add blacklist-v6 2a00:46e0::/32
|
||||
add blacklist-v6 2a00:a300::/32
|
||||
add blacklist-v6 2a00:b4c0::/32
|
||||
add blacklist-v6 2a00:bdc0:8000::/34
|
||||
add blacklist-v6 2a00:bdc0::/33
|
||||
add blacklist-v6 2a00:bdc0:c000::/35
|
||||
add blacklist-v6 2a00:bdc0:e002::/48
|
||||
add blacklist-v6 2a00:bdc0:e003::/48
|
||||
add blacklist-v6 2a00:bdc0:e004::/48
|
||||
add blacklist-v6 2a00:bdc0:e005::/48
|
||||
add blacklist-v6 2a00:bdc0:e007::/48
|
||||
add blacklist-v6 2a00:bdc0:f000::/36
|
||||
add blacklist-v6 2a00:bdc1::/32
|
||||
add blacklist-v6 2a00:bdc2::/31
|
||||
add blacklist-v6 2a00:bdc4::/30
|
||||
add blacklist-v6 2a0c:a9c7:156::/48
|
||||
add blacklist-v6 2a0c:a9c7:157::/48
|
||||
add blacklist-v6 2a0c:a9c7:158::/48
|
||||
add blacklist-v6 2a14:25c0::/32
|
||||
add blacklist-v6 2a14:25c5::/32
|
||||
add blacklist-v6 2a14:25c6::/32
|
||||
add blacklist-v6 2a14:25c7::/32
|
||||
1212
blacklists_iptables/blacklist.ipset
Normal file
1212
blacklists_iptables/blacklist.ipset
Normal file
File diff suppressed because it is too large
Load Diff
302
blacklists_nginx/README.md
Normal file
302
blacklists_nginx/README.md
Normal file
@@ -0,0 +1,302 @@
|
||||
# Nginx Blacklist Configurations
|
||||
|
||||
Auto-generated nginx configuration files for blocking networks and IP addresses.
|
||||
|
||||
## Available Files
|
||||
|
||||
### Mixed IPv4/IPv6
|
||||
|
||||
- **`blacklist.conf`** - Contains both IPv4 and IPv6 deny rules (809 entries)
|
||||
|
||||
### IPv4 Only
|
||||
|
||||
- **`blacklist-v4.conf`** - Contains only IPv4 deny rules (806 entries)
|
||||
|
||||
### IPv6 Only
|
||||
|
||||
- **`blacklist-v6.conf`** - Contains only IPv6 deny rules (3 entries)
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Usage
|
||||
|
||||
Include the desired configuration file in your nginx `server` or `location` block:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
|
||||
# Include the blacklist
|
||||
include /path/to/blacklist.conf;
|
||||
|
||||
location / {
|
||||
# your configuration
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Separate IPv4/IPv6 Files
|
||||
|
||||
For more granular control, use separate files:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
server_name example.com;
|
||||
|
||||
# Include both IPv4 and IPv6 blacklists
|
||||
include /path/to/blacklist-v4.conf;
|
||||
include /path/to/blacklist-v6.conf;
|
||||
|
||||
location / {
|
||||
# your configuration
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### HTTP Block Level
|
||||
|
||||
Apply the blacklist globally to all virtual hosts:
|
||||
|
||||
```nginx
|
||||
http {
|
||||
# Apply blacklist globally
|
||||
include /path/to/blacklist.conf;
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
# ...
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name another.com;
|
||||
# ...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Location Block Level
|
||||
|
||||
For selective blocking within specific locations:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
|
||||
location /admin {
|
||||
# Apply blacklist only to admin area
|
||||
include /path/to/blacklist.conf;
|
||||
# ...
|
||||
}
|
||||
|
||||
location /public {
|
||||
# Public area without blacklist
|
||||
# ...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Testing Configuration
|
||||
|
||||
After adding the blacklist, always test your nginx configuration:
|
||||
|
||||
```bash
|
||||
# Test configuration
|
||||
nginx -t
|
||||
|
||||
# Reload nginx if test passes
|
||||
nginx -s reload
|
||||
# or
|
||||
systemctl reload nginx
|
||||
```
|
||||
|
||||
## Custom Response
|
||||
|
||||
By default, denied IPs receive a connection drop. To customize the response:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
|
||||
# Return custom error page
|
||||
error_page 403 /403.html;
|
||||
|
||||
include /path/to/blacklist.conf;
|
||||
|
||||
location = /403.html {
|
||||
root /usr/share/nginx/html;
|
||||
internal;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note: For large blacklists, using `deny` directives (as in these files) is more efficient than `if` statements.
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- **Deny directives** are processed in order and stop at the first match
|
||||
- For optimal performance, most frequently matched IPs should be at the top
|
||||
- Current files are sorted for consistency
|
||||
- Nginx handles hundreds of deny rules efficiently
|
||||
- For very large blacklists (10,000+ entries), consider using:
|
||||
- Nginx GeoIP2 module for geographic blocking
|
||||
- nftables/iptables at the firewall level for better performance
|
||||
- Stream module for TCP/UDP level blocking
|
||||
|
||||
## Integration Examples
|
||||
|
||||
### Docker Deployment
|
||||
|
||||
```dockerfile
|
||||
FROM nginx:alpine
|
||||
|
||||
# Copy blacklist
|
||||
COPY blacklist.conf /etc/nginx/blacklist.conf
|
||||
|
||||
# Copy nginx config that includes the blacklist
|
||||
COPY nginx.conf /etc/nginx/nginx.conf
|
||||
|
||||
EXPOSE 80 443
|
||||
CMD ["nginx", "-g", "daemon off;"]
|
||||
```
|
||||
|
||||
### Kubernetes ConfigMap
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: nginx-blacklist
|
||||
data:
|
||||
blacklist.conf: |
|
||||
# Include blacklist content here
|
||||
deny 109.124.119.88/29;
|
||||
deny 109.124.66.128/30;
|
||||
# ...
|
||||
```
|
||||
|
||||
### Automated Updates
|
||||
|
||||
Set up a cron job to automatically fetch the latest blacklist:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# /etc/cron.daily/update-nginx-blacklist
|
||||
|
||||
# Download latest blacklist
|
||||
wget -q https://raw.githubusercontent.com/C24Be/AS_Network_List/main/blacklists_nginx/blacklist.conf \
|
||||
-O /etc/nginx/blacklist.conf.new
|
||||
|
||||
# Test nginx configuration
|
||||
nginx -t -c /etc/nginx/nginx.conf
|
||||
|
||||
# If test passes, reload nginx
|
||||
if [ $? -eq 0 ]; then
|
||||
mv /etc/nginx/blacklist.conf.new /etc/nginx/blacklist.conf
|
||||
systemctl reload nginx
|
||||
echo "Blacklist updated successfully"
|
||||
else
|
||||
rm /etc/nginx/blacklist.conf.new
|
||||
echo "Nginx config test failed, blacklist not updated"
|
||||
fi
|
||||
```
|
||||
|
||||
## Logging Blocked Requests
|
||||
|
||||
To log denied requests:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
|
||||
# Custom log format for denied IPs
|
||||
log_format blocked '$remote_addr - $remote_user [$time_local] '
|
||||
'"$request" 403 0 '
|
||||
'"$http_referer" "$http_user_agent"';
|
||||
|
||||
access_log /var/log/nginx/blocked.log blocked;
|
||||
|
||||
include /path/to/blacklist.conf;
|
||||
|
||||
location / {
|
||||
# your configuration
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
Check how many IPs are being blocked:
|
||||
|
||||
```bash
|
||||
# Count deny rules
|
||||
grep -c "deny" /path/to/blacklist.conf
|
||||
|
||||
# Check blocked access logs
|
||||
tail -f /var/log/nginx/blocked.log
|
||||
|
||||
# Count blocked requests today
|
||||
grep "$(date +%d/%b/%Y)" /var/log/nginx/access.log | grep " 403 " | wc -l
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Configuration Test Fails
|
||||
|
||||
```bash
|
||||
# Check syntax
|
||||
nginx -t
|
||||
|
||||
# Check for duplicate includes
|
||||
grep -r "include.*blacklist" /etc/nginx/
|
||||
|
||||
# Verify file permissions
|
||||
ls -l /path/to/blacklist.conf
|
||||
```
|
||||
|
||||
### Legitimate Users Blocked
|
||||
|
||||
Check if their IP is in the blacklist:
|
||||
|
||||
```bash
|
||||
grep "YOUR_IP" /path/to/blacklist.conf
|
||||
```
|
||||
|
||||
Whitelist specific IPs before applying the blacklist:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
|
||||
# Whitelist before blacklist
|
||||
allow 192.168.1.100; # Trusted IP
|
||||
|
||||
# Then apply blacklist
|
||||
include /path/to/blacklist.conf;
|
||||
|
||||
# Deny all others not explicitly allowed
|
||||
# deny all; # Optional
|
||||
}
|
||||
```
|
||||
|
||||
## Automatic Updates
|
||||
|
||||
These files are automatically regenerated daily when the blacklists are updated via the GitHub Actions workflow.
|
||||
|
||||
## Source
|
||||
|
||||
Generated from the blacklist files in the `blacklists/` directory by `blacklists_updater_nginx.sh`.
|
||||
|
||||
## See Also
|
||||
|
||||
- [IPTables/IPSet Format](../blacklists_iptables/README.md) - For firewall-level blocking
|
||||
- [Text Format](../blacklists/README.md) - For custom integrations
|
||||
- [Main Repository](https://github.com/C24Be/AS_Network_List) - Complete documentation
|
||||
1174
blacklists_nginx/blacklist-v4.conf
Normal file
1174
blacklists_nginx/blacklist-v4.conf
Normal file
File diff suppressed because it is too large
Load Diff
34
blacklists_nginx/blacklist-v6.conf
Normal file
34
blacklists_nginx/blacklist-v6.conf
Normal file
@@ -0,0 +1,34 @@
|
||||
# Nginx blacklist configuration (IPv6 only)
|
||||
# Auto-generated from blacklist-v6.txt
|
||||
# Last updated: 2025-12-31 06:24:51 UTC
|
||||
#
|
||||
# Usage: Include this file in your nginx server or location block:
|
||||
# include /path/to/blacklist-v6.conf;
|
||||
#
|
||||
|
||||
deny 2a00:1148::/29;
|
||||
deny 2a00:1148::/32;
|
||||
deny 2a00:46e0:2::/48;
|
||||
deny 2a00:46e0::/32;
|
||||
deny 2a00:a300::/32;
|
||||
deny 2a00:b4c0::/32;
|
||||
deny 2a00:bdc0:8000::/34;
|
||||
deny 2a00:bdc0::/33;
|
||||
deny 2a00:bdc0:c000::/35;
|
||||
deny 2a00:bdc0:e002::/48;
|
||||
deny 2a00:bdc0:e003::/48;
|
||||
deny 2a00:bdc0:e004::/48;
|
||||
deny 2a00:bdc0:e005::/48;
|
||||
deny 2a00:bdc0:e007::/48;
|
||||
deny 2a00:bdc0:f000::/36;
|
||||
deny 2a00:bdc1::/32;
|
||||
deny 2a00:bdc2::/31;
|
||||
deny 2a00:bdc4::/30;
|
||||
deny 2a0c:a9c7:156::/48;
|
||||
deny 2a0c:a9c7:157::/48;
|
||||
deny 2a0c:a9c7:158::/48;
|
||||
deny 2a14:25c0::/32;
|
||||
deny 2a14:25c5::/32;
|
||||
deny 2a14:25c6::/32;
|
||||
deny 2a14:25c7::/32;
|
||||
|
||||
1199
blacklists_nginx/blacklist.conf
Normal file
1199
blacklists_nginx/blacklist.conf
Normal file
File diff suppressed because it is too large
Load Diff
@@ -1,46 +1,19 @@
|
||||
#!/bin/sh
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
|
||||
# Input files (generated by blacklists_updater_txt.sh)
|
||||
blacklist_file="${SCRIPT_DIR}/blacklists/blacklist.txt"
|
||||
blacklist_v4_file="${SCRIPT_DIR}/blacklists/blacklist-v4.txt"
|
||||
blacklist_v6_file="${SCRIPT_DIR}/blacklists/blacklist-v6.txt"
|
||||
|
||||
# Source files for name-based VK filtering
|
||||
auto_all_v4_file="${SCRIPT_DIR}/auto/all-ru-ipv4.txt"
|
||||
auto_all_v6_file="${SCRIPT_DIR}/auto/all-ru-ipv6.txt"
|
||||
auto_ripe_v4_file="${SCRIPT_DIR}/auto/ripe-ru-ipv4.txt"
|
||||
vk_name_pattern='VK[[:space:]-]*CLOUD|VKCOMPANY|VKONTAKTE'
|
||||
|
||||
# Additional VK-only text blacklists
|
||||
blacklist_vk_file="${SCRIPT_DIR}/blacklists/blacklist-vk.txt"
|
||||
blacklist_vk_v4_file="${SCRIPT_DIR}/blacklists/blacklist-vk-v4.txt"
|
||||
blacklist_vk_v6_file="${SCRIPT_DIR}/blacklists/blacklist-vk-v6.txt"
|
||||
blacklist_file="blacklists/blacklist.txt"
|
||||
blacklist_v4_file="blacklists/blacklist-v4.txt"
|
||||
blacklist_v6_file="blacklists/blacklist-v6.txt"
|
||||
|
||||
# Output directory and files
|
||||
iptables_output_dir="${SCRIPT_DIR}/blacklists_iptables"
|
||||
iptables_output_dir="blacklists_iptables"
|
||||
iptables_output_file="${iptables_output_dir}/blacklist.ipset"
|
||||
iptables_v4_output_file="${iptables_output_dir}/blacklist-v4.ipset"
|
||||
iptables_v6_output_file="${iptables_output_dir}/blacklist-v6.ipset"
|
||||
iptables_vk_output_file="${iptables_output_dir}/blacklist-vk.ipset"
|
||||
iptables_vk_v4_output_file="${iptables_output_dir}/blacklist-vk-v4.ipset"
|
||||
iptables_vk_v6_output_file="${iptables_output_dir}/blacklist-vk-v6.ipset"
|
||||
|
||||
# Create iptables directory if it doesn't exist
|
||||
mkdir -p "${iptables_output_dir}"
|
||||
|
||||
# Build additional VK-only blacklist from network names in auto/*.txt files
|
||||
tmp_vk_file="$(mktemp "${SCRIPT_DIR}/blacklists/.blacklist-vk.XXXXXX")"
|
||||
for source_file in "${auto_all_v4_file}" "${auto_all_v6_file}" "${auto_ripe_v4_file}"; do
|
||||
[ -f "${source_file}" ] || continue
|
||||
awk -v pattern="${vk_name_pattern}" 'BEGIN { IGNORECASE = 1 } $0 ~ pattern { print $1 }' "${source_file}" >> "${tmp_vk_file}"
|
||||
done
|
||||
sort -u "${tmp_vk_file}" > "${blacklist_vk_file}"
|
||||
grep ':' "${blacklist_vk_file}" | sort -u > "${blacklist_vk_v6_file}" || true
|
||||
grep -v ':' "${blacklist_vk_file}" | sort -u > "${blacklist_vk_v4_file}" || true
|
||||
rm -f "${tmp_vk_file}"
|
||||
|
||||
# Function to generate ipset config from input file
|
||||
generate_ipset_config() {
|
||||
local input_file="$1"
|
||||
@@ -48,19 +21,6 @@ generate_ipset_config() {
|
||||
local ip_version="$3"
|
||||
local set_name="$4"
|
||||
local family="$5"
|
||||
local iptables_cmd="iptables"
|
||||
local rule_primary=""
|
||||
local rule_secondary=""
|
||||
|
||||
[ "${family}" = "inet6" ] && iptables_cmd="ip6tables"
|
||||
|
||||
if printf "%s" "${set_name}" | grep -q '^blacklist-vk'; then
|
||||
rule_primary="${iptables_cmd} -I OUTPUT -m set --match-set ${set_name} dst -j REJECT"
|
||||
rule_secondary="${iptables_cmd} -I FORWARD -m set --match-set ${set_name} dst -j REJECT"
|
||||
else
|
||||
rule_primary="${iptables_cmd} -I INPUT -m set --match-set ${set_name} src -m conntrack --ctstate NEW -j DROP"
|
||||
rule_secondary="${iptables_cmd} -I FORWARD -m set --match-set ${set_name} src -m conntrack --ctstate NEW -j DROP"
|
||||
fi
|
||||
|
||||
# Count entries for hash size calculation
|
||||
local count=$(wc -l < "${input_file}" | tr -d ' ')
|
||||
@@ -78,8 +38,8 @@ generate_ipset_config() {
|
||||
# ipset restore < $(basename ${output_file})
|
||||
#
|
||||
# 2. Use with iptables/ip6tables:
|
||||
# ${rule_primary}
|
||||
${rule_secondary:+# ${rule_secondary}}
|
||||
# iptables -I INPUT -m set --match-set ${set_name} src -m conntrack --ctstate NEW -j DROP
|
||||
# iptables -I FORWARD -m set --match-set ${set_name} src -m conntrack --ctstate NEW -j DROP
|
||||
#
|
||||
# 3. To flush/delete the set:
|
||||
# ipset flush ${set_name}
|
||||
@@ -103,8 +63,6 @@ EOF
|
||||
# Generate ipset configurations from blacklist files
|
||||
generate_ipset_config "${blacklist_v4_file}" "${iptables_v4_output_file}" "(IPv4 only)" "blacklist-v4" "inet"
|
||||
generate_ipset_config "${blacklist_v6_file}" "${iptables_v6_output_file}" "(IPv6 only)" "blacklist-v6" "inet6"
|
||||
generate_ipset_config "${blacklist_vk_v4_file}" "${iptables_vk_v4_output_file}" "(VK names, IPv4 only)" "blacklist-vk-v4" "inet"
|
||||
generate_ipset_config "${blacklist_vk_v6_file}" "${iptables_vk_v6_output_file}" "(VK names, IPv6 only)" "blacklist-vk-v6" "inet6"
|
||||
|
||||
# For mixed file, we need to create two sets (IPv4 and IPv6) as ipset doesn't support mixed families
|
||||
cat > "${iptables_output_file}" << EOF
|
||||
@@ -136,43 +94,3 @@ tail -n +2 "${iptables_v6_output_file}" | grep -E "^(create|add)" >> "${iptables
|
||||
|
||||
echo "✓ Generated (mixed IPv4/IPv6): ${iptables_output_file}"
|
||||
echo " Total entries: $(wc -l < "${blacklist_file}" | tr -d ' ')"
|
||||
|
||||
# Generate mixed VK-only ipset file (contains both v4 and v6 sets)
|
||||
cat > "${iptables_vk_output_file}" << EOF
|
||||
# IPSet blacklist configuration (VK names: VK Cloud / VKCOMPANY / VKONTAKTE)
|
||||
# Auto-generated from name-filtered auto/*.txt sources
|
||||
# Last updated: $(date -u +"%Y-%m-%d %H:%M:%S UTC")
|
||||
#
|
||||
# Usage:
|
||||
# 1. Load the ipset:
|
||||
# ipset restore < $(basename "${iptables_vk_output_file}")
|
||||
#
|
||||
# 2. Use with iptables/ip6tables:
|
||||
# iptables -I OUTPUT -m set --match-set blacklist-vk-v4 dst -j REJECT
|
||||
# iptables -I FORWARD -m set --match-set blacklist-vk-v4 dst -j REJECT
|
||||
# ip6tables -I OUTPUT -m set --match-set blacklist-vk-v6 dst -j REJECT
|
||||
# ip6tables -I FORWARD -m set --match-set blacklist-vk-v6 dst -j REJECT
|
||||
#
|
||||
# 3. To flush/delete the sets:
|
||||
# ipset flush blacklist-vk-v4 && ipset destroy blacklist-vk-v4
|
||||
# ipset flush blacklist-vk-v6 && ipset destroy blacklist-vk-v6
|
||||
#
|
||||
|
||||
EOF
|
||||
|
||||
tail -n +2 "${iptables_vk_v4_output_file}" | grep -E "^(create|add)" >> "${iptables_vk_output_file}"
|
||||
echo "" >> "${iptables_vk_output_file}"
|
||||
tail -n +2 "${iptables_vk_v6_output_file}" | grep -E "^(create|add)" >> "${iptables_vk_output_file}"
|
||||
|
||||
echo "✓ Generated (VK names, mixed IPv4/IPv6): ${iptables_vk_output_file}"
|
||||
echo " Total entries: $(wc -l < "${blacklist_vk_file}" | tr -d ' ')"
|
||||
|
||||
echo ""
|
||||
echo "VK outgoing block examples (iptables/ipset):"
|
||||
echo " ipset restore < ${iptables_vk_output_file}"
|
||||
echo " iptables -I OUTPUT -m set --match-set blacklist-vk-v4 dst -j REJECT"
|
||||
echo " iptables -I FORWARD -m set --match-set blacklist-vk-v4 dst -j REJECT"
|
||||
echo " ip6tables -I OUTPUT -m set --match-set blacklist-vk-v6 dst -j REJECT"
|
||||
echo " ip6tables -I FORWARD -m set --match-set blacklist-vk-v6 dst -j REJECT"
|
||||
echo ""
|
||||
echo "Tip: Do not install Messenger MAX on the same phone/device that has VPN access configured."
|
||||
|
||||
@@ -1,84 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Generates nftables blacklist configurations from the main blacklist
|
||||
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
INPUT_FILE="$SCRIPT_DIR/blacklists/blacklist.txt"
|
||||
OUTPUT_DIR="$SCRIPT_DIR/blacklists_nftables"
|
||||
|
||||
# Source files for name-based VK filtering
|
||||
AUTO_ALL_V4_FILE="$SCRIPT_DIR/auto/all-ru-ipv4.txt"
|
||||
AUTO_ALL_V6_FILE="$SCRIPT_DIR/auto/all-ru-ipv6.txt"
|
||||
AUTO_RIPE_V4_FILE="$SCRIPT_DIR/auto/ripe-ru-ipv4.txt"
|
||||
VK_NAME_PATTERN='VK[[:space:]-]*CLOUD|VKCOMPANY|VKONTAKTE'
|
||||
|
||||
# Additional VK-only text blacklists
|
||||
VK_INPUT_FILE="$SCRIPT_DIR/blacklists/blacklist-vk.txt"
|
||||
VK_INPUT_V4_FILE="$SCRIPT_DIR/blacklists/blacklist-vk-v4.txt"
|
||||
VK_INPUT_V6_FILE="$SCRIPT_DIR/blacklists/blacklist-vk-v6.txt"
|
||||
|
||||
# Create output directory if it doesn't exist
|
||||
mkdir -p "$OUTPUT_DIR"
|
||||
|
||||
echo "Generating nftables blacklists..."
|
||||
|
||||
# Build additional VK-only blacklist from network names in auto/*.txt files
|
||||
TMP_VK_FILE="$(mktemp "$SCRIPT_DIR/blacklists/.blacklist-vk.XXXXXX")"
|
||||
for source_file in "$AUTO_ALL_V4_FILE" "$AUTO_ALL_V6_FILE" "$AUTO_RIPE_V4_FILE"; do
|
||||
[[ -f "$source_file" ]] || continue
|
||||
awk -v pattern="$VK_NAME_PATTERN" 'BEGIN { IGNORECASE = 1 } $0 ~ pattern { print $1 }' "$source_file" >> "$TMP_VK_FILE"
|
||||
done
|
||||
sort -u "$TMP_VK_FILE" > "$VK_INPUT_FILE"
|
||||
grep ':' "$VK_INPUT_FILE" | sort -u > "$VK_INPUT_V6_FILE" || true
|
||||
grep -v ':' "$VK_INPUT_FILE" | sort -u > "$VK_INPUT_V4_FILE" || true
|
||||
rm -f "$TMP_VK_FILE"
|
||||
|
||||
# Generate mixed IPv4/IPv6 blacklist
|
||||
python3 "$SCRIPT_DIR/generate_nft_blacklist.py" \
|
||||
"$INPUT_FILE" \
|
||||
"$OUTPUT_DIR/blacklist.nft"
|
||||
|
||||
# Generate IPv4-only blacklist
|
||||
TMP_V4_FILE="/tmp/blacklist-v4.txt"
|
||||
TMP_V6_FILE="/tmp/blacklist-v6.txt"
|
||||
grep -E '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' "$INPUT_FILE" > "$TMP_V4_FILE" || true
|
||||
python3 "$SCRIPT_DIR/generate_nft_blacklist.py" \
|
||||
"$TMP_V4_FILE" \
|
||||
"$OUTPUT_DIR/blacklist-v4.nft"
|
||||
|
||||
# Generate IPv6-only blacklist
|
||||
grep -E '^[0-9a-fA-F:]+:' "$INPUT_FILE" > "$TMP_V6_FILE" || true
|
||||
python3 "$SCRIPT_DIR/generate_nft_blacklist.py" \
|
||||
"$TMP_V6_FILE" \
|
||||
"$OUTPUT_DIR/blacklist-v6.nft"
|
||||
|
||||
# Generate VK-only blacklists (network names: VK Cloud / VKCOMPANY / VKONTAKTE)
|
||||
python3 "$SCRIPT_DIR/generate_nft_blacklist.py" \
|
||||
"$VK_INPUT_FILE" \
|
||||
"$OUTPUT_DIR/blacklist-vk.nft"
|
||||
python3 "$SCRIPT_DIR/generate_nft_blacklist.py" \
|
||||
"$VK_INPUT_V4_FILE" \
|
||||
"$OUTPUT_DIR/blacklist-vk-v4.nft"
|
||||
python3 "$SCRIPT_DIR/generate_nft_blacklist.py" \
|
||||
"$VK_INPUT_V6_FILE" \
|
||||
"$OUTPUT_DIR/blacklist-vk-v6.nft"
|
||||
|
||||
# Clean up temp files
|
||||
rm -f "$TMP_V4_FILE" "$TMP_V6_FILE"
|
||||
|
||||
echo "nftables blacklists generated successfully!"
|
||||
echo ""
|
||||
echo "VM incoming block examples (all lists, nftables):"
|
||||
echo " sudo nft -f $OUTPUT_DIR/blacklist.nft"
|
||||
echo " sudo nft add chain inet filter input '{ type filter hook input priority 0; policy accept; }'"
|
||||
echo " sudo nft add rule inet filter input ip saddr @blacklist_v4 counter reject"
|
||||
echo " sudo nft add rule inet filter input ip6 saddr @blacklist_v6 counter reject"
|
||||
echo ""
|
||||
echo "VK outbound block examples for VPN clients via NAT (nftables):"
|
||||
echo " sudo nft -f $OUTPUT_DIR/blacklist-vk.nft"
|
||||
echo " sudo nft add chain inet filter forward '{ type filter hook forward priority 0; policy accept; }'"
|
||||
echo " sudo nft add rule inet filter forward iifname \"<VPN_IFACE>\" ip daddr @blacklist_v4 counter reject"
|
||||
echo " sudo nft add rule inet filter forward iifname \"<VPN_IFACE>\" ip6 daddr @blacklist_v6 counter reject"
|
||||
echo ""
|
||||
echo "Tip: Do not install Messenger MAX on the same phone/device that has VPN access configured."
|
||||
@@ -1,78 +0,0 @@
|
||||
#!/bin/sh
|
||||
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
|
||||
# Source files for name-based VK filtering
|
||||
AUTO_ALL_V4_FILE="${SCRIPT_DIR}/auto/all-ru-ipv4.txt"
|
||||
AUTO_ALL_V6_FILE="${SCRIPT_DIR}/auto/all-ru-ipv6.txt"
|
||||
AUTO_RIPE_V4_FILE="${SCRIPT_DIR}/auto/ripe-ru-ipv4.txt"
|
||||
VK_NAME_PATTERN='VK[[:space:]-]*CLOUD|VKCOMPANY|VKONTAKTE'
|
||||
|
||||
# Additional VK-only text blacklists
|
||||
VK_INPUT_FILE="${SCRIPT_DIR}/blacklists/blacklist-vk.txt"
|
||||
VK_INPUT_V4_FILE="${SCRIPT_DIR}/blacklists/blacklist-vk-v4.txt"
|
||||
VK_INPUT_V6_FILE="${SCRIPT_DIR}/blacklists/blacklist-vk-v6.txt"
|
||||
|
||||
# Output directory and files
|
||||
ROUTES_OUTPUT_DIR="${SCRIPT_DIR}/blacklists_route"
|
||||
ROUTES_V4_FILE="${ROUTES_OUTPUT_DIR}/blacklist-vk-v4.routes"
|
||||
ROUTES_V6_FILE="${ROUTES_OUTPUT_DIR}/blacklist-vk-v6.routes"
|
||||
|
||||
mkdir -p "${ROUTES_OUTPUT_DIR}" "${SCRIPT_DIR}/blacklists"
|
||||
|
||||
echo "Generating VK route blacklists..."
|
||||
|
||||
# Build additional VK-only blacklist from network names in auto/*.txt files
|
||||
TMP_VK_FILE="$(mktemp "${SCRIPT_DIR}/blacklists/.blacklist-vk.XXXXXX")"
|
||||
for source_file in "${AUTO_ALL_V4_FILE}" "${AUTO_ALL_V6_FILE}" "${AUTO_RIPE_V4_FILE}"; do
|
||||
[ -f "${source_file}" ] || continue
|
||||
awk -v pattern="${VK_NAME_PATTERN}" 'BEGIN { IGNORECASE = 1 } $0 ~ pattern { print $1 }' "${source_file}" >> "${TMP_VK_FILE}"
|
||||
done
|
||||
|
||||
sort -u "${TMP_VK_FILE}" > "${VK_INPUT_FILE}"
|
||||
grep ':' "${VK_INPUT_FILE}" | sort -u > "${VK_INPUT_V6_FILE}" || true
|
||||
grep -v ':' "${VK_INPUT_FILE}" | sort -u > "${VK_INPUT_V4_FILE}" || true
|
||||
rm -f "${TMP_VK_FILE}"
|
||||
|
||||
# Generate IPv4 routes file (route VK prefixes to loopback via 127.0.0.1)
|
||||
cat > "${ROUTES_V4_FILE}" << EOF
|
||||
# Linux routes for VK networks (IPv4)
|
||||
# Auto-generated by $(basename "$0")
|
||||
# Last updated: $(date -u +"%Y-%m-%d %H:%M:%S UTC")
|
||||
#
|
||||
# Apply:
|
||||
# sudo sh $(basename "${ROUTES_V4_FILE}")
|
||||
#
|
||||
|
||||
EOF
|
||||
|
||||
while IFS= read -r network; do
|
||||
[ -n "${network}" ] || continue
|
||||
printf 'ip route replace %s via 127.0.0.1 dev lo onlink\n' "${network}" >> "${ROUTES_V4_FILE}"
|
||||
done < "${VK_INPUT_V4_FILE}"
|
||||
|
||||
# Generate IPv6 routes file (route VK prefixes to loopback via ::1)
|
||||
cat > "${ROUTES_V6_FILE}" << EOF
|
||||
# Linux routes for VK networks (IPv6)
|
||||
# Auto-generated by $(basename "$0")
|
||||
# Last updated: $(date -u +"%Y-%m-%d %H:%M:%S UTC")
|
||||
#
|
||||
# Apply:
|
||||
# sudo sh $(basename "${ROUTES_V6_FILE}")
|
||||
#
|
||||
|
||||
EOF
|
||||
|
||||
while IFS= read -r network; do
|
||||
[ -n "${network}" ] || continue
|
||||
printf 'ip -6 route replace %s via ::1 dev lo\n' "${network}" >> "${ROUTES_V6_FILE}"
|
||||
done < "${VK_INPUT_V6_FILE}"
|
||||
|
||||
echo "✓ Generated: ${ROUTES_V4_FILE} (entries: $(wc -l < "${VK_INPUT_V4_FILE}" | tr -d ' '))"
|
||||
echo "✓ Generated: ${ROUTES_V6_FILE} (entries: $(wc -l < "${VK_INPUT_V6_FILE}" | tr -d ' '))"
|
||||
echo ""
|
||||
echo "Examples:"
|
||||
echo " sudo sh ${ROUTES_V4_FILE}"
|
||||
echo " sudo sh ${ROUTES_V6_FILE}"
|
||||
@@ -1,113 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
check_nft_blacklist.py
|
||||
Checks if an IP address is in the nftables blacklist configuration.
|
||||
Usage:
|
||||
check_nft_blacklist.py nft_bl.conf 192.168.1.1
|
||||
check_nft_blacklist.py nft_bl.conf 2001:db8::1
|
||||
"""
|
||||
|
||||
import sys
|
||||
import re
|
||||
from ipaddress import ip_address, ip_network, AddressValueError
|
||||
from pathlib import Path
|
||||
|
||||
def parse_nft_config(config_path):
|
||||
"""Extract IPv4 and IPv6 prefixes from nftables config."""
|
||||
p = Path(config_path)
|
||||
if not p.exists():
|
||||
raise FileNotFoundError(f"Config file not found: {config_path}")
|
||||
|
||||
content = p.read_text(encoding="utf-8")
|
||||
v4_prefixes = []
|
||||
v6_prefixes = []
|
||||
|
||||
# Parse IPv4 set (blacklist_v4)
|
||||
v4_match = re.search(
|
||||
r'set blacklist_v4\s*\{[^}]*elements\s*=\s*\{([^}]+)\}',
|
||||
content,
|
||||
re.DOTALL
|
||||
)
|
||||
if v4_match:
|
||||
elements = v4_match.group(1)
|
||||
# Extract all CIDR notations
|
||||
for match in re.finditer(r'(\d+\.\d+\.\d+\.\d+(?:/\d+)?)', elements):
|
||||
try:
|
||||
v4_prefixes.append(ip_network(match.group(1), strict=False))
|
||||
except Exception as e:
|
||||
print(f"Warning: Could not parse IPv4 prefix '{match.group(1)}': {e}", file=sys.stderr)
|
||||
|
||||
# Parse IPv6 set (blacklist_v6)
|
||||
v6_match = re.search(
|
||||
r'set blacklist_v6\s*\{[^}]*elements\s*=\s*\{([^}]+)\}',
|
||||
content,
|
||||
re.DOTALL
|
||||
)
|
||||
if v6_match:
|
||||
elements = v6_match.group(1)
|
||||
# Extract all IPv6 CIDR notations
|
||||
for match in re.finditer(r'([0-9a-fA-F:]+(?:/\d+)?)', elements):
|
||||
try:
|
||||
v6_prefixes.append(ip_network(match.group(1), strict=False))
|
||||
except Exception as e:
|
||||
# Skip false matches from comments or other text
|
||||
pass
|
||||
|
||||
return v4_prefixes, v6_prefixes
|
||||
|
||||
def check_ip_in_blacklist(ip_addr, v4_prefixes, v6_prefixes):
|
||||
"""Check if IP address is in any of the blacklist prefixes."""
|
||||
try:
|
||||
addr = ip_address(ip_addr)
|
||||
except AddressValueError as e:
|
||||
raise ValueError(f"Invalid IP address: {ip_addr} ({e})")
|
||||
|
||||
prefixes = v4_prefixes if addr.version == 4 else v6_prefixes
|
||||
|
||||
for prefix in prefixes:
|
||||
if addr in prefix:
|
||||
return True, prefix
|
||||
|
||||
return False, None
|
||||
|
||||
def main(argv):
|
||||
if len(argv) < 3:
|
||||
print("Usage: python3 check_nft_blacklist.py <nft_config.conf> <ip_address>")
|
||||
print("Examples:")
|
||||
print(" check_nft_blacklist.py nft_bl.conf 192.168.1.1")
|
||||
print(" check_nft_blacklist.py nft_bl.conf 2001:db8::1")
|
||||
return 2
|
||||
|
||||
config_file = argv[1]
|
||||
ip_to_check = argv[2]
|
||||
|
||||
# Parse the nftables config
|
||||
try:
|
||||
print(f"Loading blacklist from: {config_file}")
|
||||
v4_prefixes, v6_prefixes = parse_nft_config(config_file)
|
||||
print(f"Loaded {len(v4_prefixes)} IPv4 prefixes and {len(v6_prefixes)} IPv6 prefixes")
|
||||
except Exception as e:
|
||||
print(f"ERROR: Could not parse config file: {e}", file=sys.stderr)
|
||||
return 3
|
||||
|
||||
# Check if IP is in blacklist
|
||||
try:
|
||||
is_blocked, matching_prefix = check_ip_in_blacklist(ip_to_check, v4_prefixes, v6_prefixes)
|
||||
|
||||
print(f"\nChecking IP: {ip_to_check}")
|
||||
print("-" * 50)
|
||||
|
||||
if is_blocked:
|
||||
print(f"✗ BLOCKED - IP is in blacklist")
|
||||
print(f" Matching prefix: {matching_prefix}")
|
||||
return 1
|
||||
else:
|
||||
print(f"✓ OK - IP is NOT in blacklist")
|
||||
return 0
|
||||
|
||||
except ValueError as e:
|
||||
print(f"ERROR: {e}", file=sys.stderr)
|
||||
return 4
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -1,155 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
generate_nft_blacklist.py
|
||||
Reads prefixes from a file or stdin, aggregates them and writes nftables config.
|
||||
Uses named sets for efficient blacklist management.
|
||||
Usage:
|
||||
git clone https://github.com/C24Be/AS_Network_List.git
|
||||
generate_nft_blacklist.py ./AS_Network_List/blacklists/blacklist.txt nft_bl.conf
|
||||
cp nft_bl.conf /etc/nftables.d/
|
||||
systemctl restart nftables
|
||||
"""
|
||||
|
||||
import sys
|
||||
from ipaddress import ip_network, collapse_addresses
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
|
||||
def read_lines(path_or_dash):
|
||||
if path_or_dash == "-":
|
||||
print("Reading prefixes from STDIN...")
|
||||
return [ln.rstrip("\n") for ln in sys.stdin]
|
||||
p = Path(path_or_dash)
|
||||
if not p.exists():
|
||||
raise FileNotFoundError(f"Input file not found: {path_or_dash}")
|
||||
text = p.read_text(encoding="utf-8")
|
||||
return text.splitlines()
|
||||
|
||||
def aggregate_prefixes(lines):
|
||||
v4, v6, invalid = [], [], []
|
||||
for lineno, ln in enumerate(lines, start=1):
|
||||
s = ln.strip()
|
||||
if not s or s.startswith("#"):
|
||||
continue
|
||||
try:
|
||||
net = ip_network(s, strict=False)
|
||||
if net.version == 4:
|
||||
v4.append(net)
|
||||
else:
|
||||
v6.append(net)
|
||||
except Exception as e:
|
||||
invalid.append((lineno, s, str(e)))
|
||||
agg_v4 = list(collapse_addresses(sorted(v4, key=lambda x: (int(x.network_address), x.prefixlen))))
|
||||
agg_v6 = list(collapse_addresses(sorted(v6, key=lambda x: (int(x.network_address), x.prefixlen))))
|
||||
return agg_v4, agg_v6, invalid
|
||||
|
||||
def make_nft_config(agg_v4, agg_v6, comment=None):
|
||||
lines = []
|
||||
lines.append("# Autogenerated nftables blacklist")
|
||||
lines.append(f"# Generated: {datetime.utcnow().isoformat()}Z")
|
||||
if comment:
|
||||
lines.append(f"# {comment}")
|
||||
lines.append(f"# IPv4: {len(agg_v4)}, IPv6: {len(agg_v6)}")
|
||||
lines.append("")
|
||||
lines.append("table inet filter {")
|
||||
lines.append("")
|
||||
|
||||
# Define IPv4 blacklist set
|
||||
lines.append(" set blacklist_v4 {")
|
||||
lines.append(" type ipv4_addr")
|
||||
lines.append(" flags interval")
|
||||
if agg_v4:
|
||||
lines.append(" elements = {")
|
||||
for i, net in enumerate(agg_v4):
|
||||
comma = "," if i < len(agg_v4) - 1 else ""
|
||||
lines.append(f" {net.with_prefixlen}{comma}")
|
||||
lines.append(" }")
|
||||
lines.append(" }")
|
||||
lines.append("")
|
||||
|
||||
# Define IPv6 blacklist set
|
||||
lines.append(" set blacklist_v6 {")
|
||||
lines.append(" type ipv6_addr")
|
||||
lines.append(" flags interval")
|
||||
if agg_v6:
|
||||
lines.append(" elements = {")
|
||||
for i, net in enumerate(agg_v6):
|
||||
comma = "," if i < len(agg_v6) - 1 else ""
|
||||
lines.append(f" {net.with_prefixlen}{comma}")
|
||||
lines.append(" }")
|
||||
lines.append(" }")
|
||||
lines.append("")
|
||||
|
||||
# Define input chain with set lookups
|
||||
lines.append(" chain input {")
|
||||
lines.append(" type filter hook input priority 0;")
|
||||
lines.append(" policy accept;")
|
||||
lines.append("")
|
||||
lines.append(" ct state { established, related } accept")
|
||||
lines.append("")
|
||||
if agg_v4:
|
||||
lines.append(" ip saddr @blacklist_v4 counter drop")
|
||||
if agg_v6:
|
||||
lines.append(" ip6 saddr @blacklist_v6 counter drop")
|
||||
lines.append(" }")
|
||||
lines.append("}")
|
||||
return "\n".join(lines)
|
||||
|
||||
def write_output(outpath, content):
|
||||
if outpath == "-":
|
||||
print(content)
|
||||
return
|
||||
p = Path(outpath)
|
||||
p.write_text(content, encoding="utf-8")
|
||||
p.chmod(0o644)
|
||||
print(f"Wrote nft config to: {p} (size: {p.stat().st_size} bytes)")
|
||||
|
||||
def main(argv):
|
||||
if len(argv) < 3:
|
||||
print("Usage: python3 generate_nft_blacklist.py input.txt output.conf")
|
||||
print("Use '-' as input or output to mean STDIN/STDOUT respectively.")
|
||||
return 2
|
||||
|
||||
infile, outfile = argv[1], argv[2]
|
||||
try:
|
||||
lines = read_lines(infile)
|
||||
except Exception as e:
|
||||
print(f"ERROR reading input: {e}", file=sys.stderr)
|
||||
return 3
|
||||
|
||||
if not any(line.strip() and not line.strip().startswith("#") for line in lines):
|
||||
print("WARNING: input contains no prefixes (empty or only comments). Nothing to aggregate.")
|
||||
nft_conf = make_nft_config([], [], comment="Empty input produced no prefixes")
|
||||
write_output(outfile, nft_conf)
|
||||
return 0
|
||||
|
||||
agg_v4, agg_v6, invalid = aggregate_prefixes(lines)
|
||||
|
||||
if invalid:
|
||||
print("Some lines could not be parsed (line, text, error):")
|
||||
for ln, txt, err in invalid:
|
||||
print(f" {ln}: '{txt}' --> {err}", file=sys.stderr)
|
||||
|
||||
print(f"Aggregated IPv4 prefixes: {len(agg_v4)}")
|
||||
for n in agg_v4:
|
||||
print(" v4:", n)
|
||||
print(f"Aggregated IPv6 prefixes: {len(agg_v6)}")
|
||||
for n in agg_v6:
|
||||
print(" v6:", n)
|
||||
|
||||
nft_conf = make_nft_config(agg_v4, agg_v6, comment=f"Source: {infile}")
|
||||
try:
|
||||
write_output(outfile, nft_conf)
|
||||
except Exception as e:
|
||||
print(f"ERROR writing output: {e}", file=sys.stderr)
|
||||
return 4
|
||||
|
||||
print("Done.")
|
||||
print("Load with: sudo nft -f <output.conf>")
|
||||
print("View counters: sudo nft list chain inet filter input -a")
|
||||
print("View sets: sudo nft list set inet filter blacklist_v4")
|
||||
print(" sudo nft list set inet filter blacklist_v6")
|
||||
return 0
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
Reference in New Issue
Block a user