24 Oct
2022
24 Oct
'22
5:01 a.m.
Hello list,
Finally have working firewall rules analysis code up
& running. This turns out to be a hard problem!
A lot of people have worked out the math, but there
are a ton of picky obstacles to applying them to real
sets of iptables rules. I hope to flesh out the code
over the next few months and am hoping for a release
(public open source) soon.
--
Charles Polisher
24 Oct
24 Oct
4:56 p.m.
I don't think I understand. What are you trying to do?
Are you looking at performace or load balancing?
-Gary
On Sun, Oct 23, 2022 at 10:01:02PM -0700, Charles Polisher wrote:
Hello list,
Finally have working firewall rules analysis code up
& running. This turns out to be a hard problem!
A lot of people have worked out the math, but there
are a ton of picky obstacles to applying them to real
sets of iptables rules. I hope to flesh out the code
over the next few months and am hoping for a release
(public open source) soon.
--
Charles Polisher
_______________________________________________
Lug-nuts mailing list -- lug-nuts(a)bigbrie.com
To unsubscribe send an email to lug-nuts-leave(a)bigbrie.com
5:41 p.m.
On 10/24/22 09:56, Gary wrote:
I don't think I understand. What are you trying
to do?
Are you looking at performace or load balancing?
-Gary
Consistency and correctness. Firewall rules can be self-
inconsistent. Hand-audits of firewalls often show lots
of mistakes, for example this simple "shadowing":
target prot source destination
ACCEPT tcp 10.2.0.0/16 192.168.1.200 ctstate NEW tcp dpt:53 /*
shadows next rule */
ACCEPT tcp 10.2.0.1 192.168.1.200 ctstate NEW tcp dpt:53 /*
shadowed */
That second rule can't match anything, because the first rule
"shadows" it, that is matches on an equal or larger address
region. Lots of hard to analyze variations of this exist. Besides
shadowing, there are hard to suss-out classes of mistakes, like
rules that call out addresses for hosts that no longer exist,
or failure to admit packets for running services from all needed
subnets. I worked on a firewall that had a typical big ball o'
mud ruleset that had been hacked on for years -- everyone was
afraid to make changes because it was terrifically hard to tell
what the outcome would be.
6:40 p.m.
Is it some kind of a Monte Carlo simulation against a ruleset over a data space?
On Mon, Oct 24, 2022 at 10:41:26AM -0700, Charles Polisher wrote:
On 10/24/22 09:56, Gary wrote:
I don't think I understand. What are you
trying to do?
Are you looking at performace or load balancing?
-Gary
Consistency and correctness. Firewall rules can be self-
inconsistent. Hand-audits of firewalls often show lots
of mistakes, for example this simple "shadowing":
target prot source destination
ACCEPT tcp 10.2.0.0/16 192.168.1.200 ctstate NEW tcp dpt:53 /* shadows
next rule */
ACCEPT tcp 10.2.0.1 192.168.1.200 ctstate NEW tcp dpt:53 /*
shadowed */
That second rule can't match anything, because the first rule
"shadows" it, that is matches on an equal or larger address
region. Lots of hard to analyze variations of this exist. Besides
shadowing, there are hard to suss-out classes of mistakes, like
rules that call out addresses for hosts that no longer exist,
or failure to admit packets for running services from all needed
subnets. I worked on a firewall that had a typical big ball o'
mud ruleset that had been hacked on for years -- everyone was
afraid to make changes because it was terrifically hard to tell
what the outcome would be.
_______________________________________________
Lug-nuts mailing list -- lug-nuts(a)bigbrie.com
To unsubscribe send an email to lug-nuts-leave(a)bigbrie.com
8:40 p.m.
On 10/24/22 11:40, Gary wrote:
Is it some kind of a Monte Carlo simulation against a
ruleset over a data space?
Modeling it in SQL, so just a query or two. (I parse raw
iptables-save in XML format to grab the data.)
One of the big surprises was finding you can model the
address space as a square box in cartesian coordinates
(upper left x, upper left y, lower right x, lower right y)
mapped from (IP address range begin, port range begin,
IP address range end, port range end), then use conventional
logic for "does box A cover box B", which PostgreSQL has
operators for. A simplfiied sketch of the query to
discover rules that cover other rules is:
SELECT mumble
FROM rules a
JOIN rules b
ON a.rulenum < b.rulenum
AND covers(
a.srcipaddrbegin, a.srcportend, a.srcipaddrend,
a.srcportbegin,
b.srcipaddrbegin, b.srcportend, b.srcipaddrend,
b.srcportbegin
)
AND covers(
a.dstipaddrbegin, a.dstportend, a.dstipaddrend,
a.dstportbegin,
b.dstipaddrbegin, b.dstportend, b.dstipaddrend,
b.dstportbegin
);
where covers() is a function:
CREATE OR REPLACE FUNCTION covers(ulxa DOUBLE PRECISION,
ulya DOUBLE PRECISION,
lrxa DOUBLE PRECISION,
lrya DOUBLE PRECISION,
ulxb DOUBLE PRECISION,
ulyb DOUBLE PRECISION,
lrxb DOUBLE PRECISION,
lryb DOUBLE PRECISION
)
RETURNS BOOLEAN
AS $$
-- Interpret two (IP address:port) ranges
-- geometrically, return TRUE if "A" covers "B"
-- or is coincident (perfectly overlaps).
DECLARE
covered boolean;
BEGIN
SELECT INTO covered
box( point(ulxb, ulyb), point(lrxb, lryb))
<@ -- "covered by OR on"
box( point(ulxa, ulya), point(lrxa, lrya));
RETURN covered;
END;
$$ LANGUAGE plpgsql;
25 Oct
25 Oct
9:27 a.m.
Looking at things geometrically is a nice idea. It brings home to me the fact that there
is a lot of ip-space out there that I should just exclude.
-Gary
On Mon, Oct 24, 2022 at 01:40:46PM -0700, Charles Polisher wrote:
On 10/24/22 11:40, Gary wrote:
Is it some kind of a Monte Carlo simulation
against a ruleset over a data space?
Modeling it in SQL, so just a query or two. (I parse raw
iptables-save in XML format to grab the data.)
One of the big surprises was finding you can model the
address space as a square box in cartesian coordinates
(upper left x, upper left y, lower right x, lower right y)
mapped from (IP address range begin, port range begin,
IP address range end, port range end), then use conventional
logic for "does box A cover box B", which PostgreSQL has
operators for. A simplfiied sketch of the query to
discover rules that cover other rules is:
SELECT mumble
FROM rules a
JOIN rules b
ON a.rulenum < b.rulenum
AND covers(
a.srcipaddrbegin, a.srcportend, a.srcipaddrend,
a.srcportbegin,
b.srcipaddrbegin, b.srcportend, b.srcipaddrend,
b.srcportbegin
)
AND covers(
a.dstipaddrbegin, a.dstportend, a.dstipaddrend,
a.dstportbegin,
b.dstipaddrbegin, b.dstportend, b.dstipaddrend,
b.dstportbegin
);
where covers() is a function:
CREATE OR REPLACE FUNCTION covers(ulxa DOUBLE PRECISION,
ulya DOUBLE PRECISION,
lrxa DOUBLE PRECISION,
lrya DOUBLE PRECISION,
ulxb DOUBLE PRECISION,
ulyb DOUBLE PRECISION,
lrxb DOUBLE PRECISION,
lryb DOUBLE PRECISION
)
RETURNS BOOLEAN
AS $$
-- Interpret two (IP address:port) ranges
-- geometrically, return TRUE if "A" covers "B"
-- or is coincident (perfectly overlaps).
DECLARE
covered boolean;
BEGIN
SELECT INTO covered
box( point(ulxb, ulyb), point(lrxb, lryb))
<@ -- "covered by OR on"
box( point(ulxa, ulya), point(lrxa, lrya));
RETURN covered;
END;
$$ LANGUAGE plpgsql;
_______________________________________________
Lug-nuts mailing list -- lug-nuts(a)bigbrie.com
To unsubscribe send an email to lug-nuts-leave(a)bigbrie.com
3:17 p.m.
On 10/25/22 02:27, Gary wrote:
Looking at things geometrically is a nice idea. It
brings home to me the fact that there is a lot of ip-space out there that I should just
exclude.
Martians. You should exclude martians. Here's a starter set:
IPTABLES="$(which iptables) --wait "
IP6TABLES="$(which ip6tables) --wait "
# Log and filter martians/bogons (BCP 38) (TODO: improve efficiency with
hash:ip ipsets ?)
$IPTABLES -A INPUT -s 10.0.0.0/8 -m limit --limit 10/minute -j
LOG --log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian PRIV RFC 1918'
$IPTABLES -A INPUT -s
10.0.0.0/8 -j
DROP -m comment --comment 'martian PRIV
RFC 1918'
$IPTABLES -A INPUT -s 192.168.0.0/24 -m limit --limit 10/minute -j
LOG --log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian PRIV RFC 1918'
$IPTABLES -A INPUT -s
192.168.0.0/24 -j
DROP -m comment --comment 'martian PRIV
RFC 1918'
$IPTABLES -A INPUT -s 127.0.0.0/8 -m limit --limit 10/minute -j
LOG --log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian LOOPBACK'
$IPTABLES -A INPUT -s
127.0.0.0/8 -j
DROP -m comment --comment 'martian LOOPBACK'
$IPTABLES -A INPUT -s 192.0.2.0/24 -m limit --limit 10/minute -j
LOG --log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian TESTNET RFC5737'
$IPTABLES -A INPUT -s
192.0.2.0/24 -j
DROP -m comment --comment 'martian
TESTNET RFC5737'
$IPTABLES -A INPUT -s 192.31.196.0/24 -m limit --limit 10/minute -j
LOG --log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian AS112 Blackhole'
$IPTABLES -A INPUT -s
192.31.196.0/24 -j
DROP -m comment --comment 'martian AS112
Blackhole'
$IPTABLES -A INPUT -s 198.18.0.0/15 -m limit --limit 10/minute -j
LOG --log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian NETBENCH RFC2544'
$IPTABLES -A INPUT -s
198.18.0.0/15 -j
DROP -m comment --comment 'martian
NETBENCH RFC2544'
$IPTABLES -A INPUT -s 198.51.100.0/24 -m limit --limit 10/minute -j
LOG --log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian TESTNET RFC5737'
$IPTABLES -A INPUT -s
198.51.100.0/24 -j
DROP -m comment --comment 'martian
TESTNET RFC5737'
$IPTABLES -A INPUT -s 203.0.113.0/24 -m limit --limit 10/minute -j
LOG --log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian TESTNET RFC5737'
$IPTABLES -A INPUT -s
203.0.113.0/24 -j
DROP -m comment --comment 'martian
TESTNET RFC5737'
$IPTABLES -A INPUT -s 233.252.0.0/24 -m limit --limit 10/minute -j
LOG --log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian TESTNET RFC5774'
$IPTABLES -A INPUT -s
233.252.0.0/24 -j
DROP -m comment --comment 'martian
TESTNET RFC5774'
$IPTABLES -A INPUT -s 255.255.255.255/32 -m limit --limit 10/minute -j
LOG --log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian LTD BCAST RFC 919'
$IPTABLES -A INPUT -s
255.255.255.255/32 -j
DROP -m comment --comment 'martian LTD
BCAST RFC 919'
# IPv6 Martians list 2019-05-12 courtesy Job Snijders via "whois -h
whois.ripe.net fltr-martian-v6"
$IP6TABLES -A INPUT -s 0000::/8 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian loopback, unspecified, v4-mapped'
$IP6TABLES -A INPUT -s 0000::/8 -j DROP -m comment --comment
'martian loopback, unspecified, v4-mapped'
$IP6TABLES -A INPUT -s 0064:ff9b::/96 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian IPv4-IPv6 Translat. [RFC6052]'
$IP6TABLES -A INPUT -s 0064:ff9b::/96 -j DROP -m comment --comment
'martian IPv4-IPv6 Translat. [RFC6052]'
$IP6TABLES -A INPUT -s 0100::/8 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian reserved for Discard-Only Address Block [RFC6666]'
$IP6TABLES -A INPUT -s 0100::/8 -j DROP -m comment --comment
'martian reserved for Discard-Only Address Block [RFC6666]'
$IP6TABLES -A INPUT -s 0200::/7 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC4048]'
$IP6TABLES -A INPUT -s 0200::/7 -j DROP -m comment --comment
'martian Reserved by IETF [RFC4048]'
$IP6TABLES -A INPUT -s 0400::/6 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s 0400::/6 -j DROP -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s 0800::/5 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s 0800::/5 -j DROP -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s 1000::/4 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s 1000::/4 -j DROP -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s 2001::/32 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Teredo prefix [RFC4380]'
$IP6TABLES -A INPUT -s 2001::/32 -j DROP -m comment --comment
'martian Teredo prefix [RFC4380]'
$IP6TABLES -A INPUT -s 2001:0002::/48 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Benchmarking [RFC5180]'
$IP6TABLES -A INPUT -s 2001:0002::/48 -j DROP -m comment --comment
'martian Benchmarking [RFC5180]'
$IP6TABLES -A INPUT -s 2001:0003::/32 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Automatic Multicast Tunneling [RFC7450]'
$IP6TABLES -A INPUT -s 2001:0003::/32 -j DROP -m comment --comment
'martian Automatic Multicast Tunneling [RFC7450]'
$IP6TABLES -A INPUT -s 2001:10::/28 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Deprecated ORCHID [RFC4843]'
$IP6TABLES -A INPUT -s 2001:10::/28 -j DROP -m comment --comment
'martian Deprecated ORCHID [RFC4843]'
$IP6TABLES -A INPUT -s 2001:20::/28 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian ORCHIDv2 [RFC7343]'
$IP6TABLES -A INPUT -s 2001:20::/28 -j DROP -m comment --comment
'martian ORCHIDv2 [RFC7343]'
$IP6TABLES -A INPUT -s 2001:db8::/32 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian NON-ROUTABLE range to be used for documentation purpose [RFC3849]'
$IP6TABLES -A INPUT -s 2001:db8::/32 -j DROP -m comment --comment
'martian NON-ROUTABLE range to be used for documentation purpose [RFC3849]'
$IP6TABLES -A INPUT -s 2002::/16 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian 6to4 prefix [RFC3068]'
$IP6TABLES -A INPUT -s 2002::/16 -j DROP -m comment --comment
'martian 6to4 prefix [RFC3068]'
$IP6TABLES -A INPUT -s 3ffe::/16 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian used for the 6bone but was returned [RFC5156]'
$IP6TABLES -A INPUT -s 3ffe::/16 -j DROP -m comment --comment
'martian used for the 6bone but was returned [RFC5156]'
$IP6TABLES -A INPUT -s 4000::/3 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s 4000::/3 -j DROP -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s 5f00::/8 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian used for the 6bone but was returned [RFC5156]'
$IP6TABLES -A INPUT -s 5f00::/8 -j DROP -m comment --comment
'martian used for the 6bone but was returned [RFC5156]'
$IP6TABLES -A INPUT -s 6000::/3 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s 6000::/3 -j DROP -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s 8000::/3 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s 8000::/3 -j DROP -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s a000::/3 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s a000::/3 -j DROP -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s c000::/3 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC4291'
$IP6TABLES -A INPUT -s c000::/3 -j DROP -m comment --comment
'martian Reserved by IETF [RFC4291'
$IP6TABLES -A INPUT -s e000::/4 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s e000::/4 -j DROP -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s f000::/5 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s f000::/5 -j DROP -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s f800::/6 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s f800::/6 -j DROP -m comment --comment
'martian Reserved by IETF [RFC4291]'
$IP6TABLES -A INPUT -s fc00::/7 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Unique Local Unicast [RFC4193]'
$IP6TABLES -A INPUT -s fc00::/7 -j DROP -m comment --comment
'martian Unique Local Unicast [RFC4193]'
$IP6TABLES -A INPUT -s fec0::/10 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Reserved by IETF [RFC3879]'
$IP6TABLES -A INPUT -s fec0::/10 -j DROP -m comment --comment
'martian Reserved by IETF [RFC3879]'
$IP6TABLES -A INPUT -s ff00::/8 -m limit --limit 10/minute -j LOG
--log-prefix 'FW_DROP_MARTIAN ' --log-level 4 -m comment --comment
'martian Multicast [RFC4291]'
$IP6TABLES -A INPUT -s ff00::/8 -j DROP -m comment --comment
'martian Multicast [RFC4291]'
28 Oct
28 Oct
1:47 a.m.
Quoting Charles Polisher (chas(a)chasmo.org):
On 10/25/22 02:27, Gary wrote:
Hear, hear!
To clarify, for those who are unclear on the concept, a "Martian packet"
in this context is one that should never legitimately appear on the
public Internet (unless tunnelled), because it contains a source or
destination address that is reserved for special use by IANA. The
classic example is all IPs in RFC1918 RFC 5735, and RFC 6598. A
metacategory of "bogon IP" adds to that IPs from netblocks not yet
allocated by IANA or the Regional Internet Registries (ARIN, RIPE,
APNIC, etc.).
(Bogons = Martians + unallocated.)
Receiving a Martian / bogon packet means either the originating
equipment is badly malfunctioning / misconfigured _or_ someone is up to
no good.
https://www.team-cymru.com/bogon-reference
https://www.cyberciti.biz/faq/linux-log-suspicious-martian-packets-un-routa…
Looking at things geometrically is a nice idea.
It brings home to me the fact that there is a lot of ip-space out there that I should just
exclude.
Martians. You should exclude martians. Here's a starter set:
566
days inactive
570
days old
7 comments
3 participants
participants (3)
-
Charles Polisher -
Gary -
Rick Moen