In addition to our dangling SPF targets research, we also analyzed SPF records for around 21 types of problems. This article is a summary of this research from a sample of 50,000 popular domains and domains owned by Fortune 500 companies with miscellaneous problem examples. After following SPF includes and redirects we analyzed 55551 TXT records that appeared to be SPF records (from 70798 domains). The software detected over 3600 problems. Around 4.2% of these domains with SPF records had SPF problems. (A small sample with explanations is shared below.)
SPF records, as defined in RFC 7208 for the Sender Policy Framework for authorizing use of domains in emails, provide a simple way for administrators to define the networks or specific hosts that are allowed to send emails using their domain(s). (We also have tests for the deprecated RFC 4408 SPF records, RRTYPE #99, but we didn't check for them for all these domains in this research and are not included in this summary.)
Receiving email servers can use the SPF rules found in DNS TXT records to consider the policies. There are multiple software implementations for SPF parsing and policy decisions. How receiving mail servers honor the SPF depends on what and how they are used. Many mail servers don't even use SPF.
The SPF policy data is provided in a TXT record. By RFC requirements, only one TXT record should contain the SPF policies. A software implementation could fail altogether for that domain, use both (or all), or use none, for example.
scribd.com. 3600 IN TXT "v=spf1 include:_spf.google.com include:servers.mcsv.net include:mail.zendesk.com include:mailgun.org include:spf.tipalti.com include:m.lessonly.com ~all" scribd.com. 3600 IN TXT "v=spf2.0/mfrom v=spf1 include:_spf.google.com include:servers.mcsv.net include:mail.zendesk.com include:mailgun.org include:spf.tipalti.com include:m.lessonly.com ~all"(Well, that second one isn't even SPF but still has "v=spf1" intent in it; and the obsolete Sender ID also shouldn't begin with "v=".)
huffingtonpost.com. 86400 IN TXT "v=spf1 include:_ipspf.yahoo.com -all" huffingtonpost.com. 86400 IN TXT "v=spf1 mx include:_spf.createsend.com include:aspmx.sailthru.com include:_spf.google.com include:amazonses.com ip4:74.205.21.242 ip4:149.174.0.0/16 ip4:64.236.0.0/16 ip4:204.29.186.0/23 ip4:148.163.135.35/32 ip4:148.163.139.55/32 ~all"
The "ptr" mechanism is used for checking IP address mapping back to a domain name, aka reverse DNS. For over 15 years, this feature was recommended as not as reliable as other checks and for over 8 years, it was documented that it should not be used.
baidu.com. 7199 IN TXT "v=spf1 include:spf1.baidu.com include:spf2.baidu.com include:spf3.baidu.com include:spf4.baidu.com a mx ptr -all"
The SPF rules are made up of various keys delimited from their values with colons (mechanisms) or equal signs (modifiers). In all cases, the unknown modifier keys were caused by configuration mistakes or typos, such as missing or adding a space or merging another TXT record content into a single TXT record.
tencent.com. 7200 IN TXT "v=spf1 include:spf.mail.tencent.com include:spf.mail.qq.com -all apple-domain-verification=cru7fAJlGpvi4jOL"(A non-SPF key=value is included in this SPF.)
gamespot.com. 86400 IN TXT "facebook-domain-verification=fbzdquphz9r1fkwjq7toys5kwtrh9e14d15e6ed4b3400fb07d ed58eb4e6fc8google-site-verification=0Ni8kZjRQQ7k7L1ka7GUuhIAW9wKiP0yuc0s6v8UVjE v=spf1 mx ip4:64.30.224.0/20 ip4:74.125.148.0/22 ip4:216.239.112.0/20 ip4:202.73.54.176/28 ip4:58.65.7.128/28 ip4:62.108.138.0/28 ip4:198.37.150.9 include:sendgrid.net -allv=spf1 mx include:spf.protection.outlook.com ~all" gamespot.com. 86400 IN TXT "v=spf1include:spf.protection.outlook.cominclude:sendgrid.net-all"(Data is merged breaking both of these, likely non SPF for this domain.)
The unknown mechanisms are almost all caused by adding a space before rule values or by having rule values without the corresponding key. Many are also caused by other garbage included in a the SPF. A common problem is a pasted-in value that accidently chopped a key or rrdata separating it with a space.
autohome.com.cn. 600 IN TXT "v=spf1 mx ip4:103.75.152.146 ip 4:114.251.201.21 ip4:3.125.202.38 ip4:103.75.154.240 ip:114.251.201.25 include: autohome.com.cn -all"(Oops, used "ip" instead of "ip4".)
cmu.edu. 3599 IN TXT "[NRDR 31504_IN_TXT_v=spf1.include:_spf.cmu.edu.~all jxEj2x9CC1Gu8ePZldY84w]" cmu.edu. 3599 IN TXT "v=spf1 include:_spf.cmu.edu ~al l"(Well this one is not technically broken, but our SPF checker did see that first record is junk.)
Per the requirements, SPF records are defined a specific way. We found many other TXT records that were likely meant to be SPF policy rules, but are probably not used by any receiving mail server.
jeep.com. 3600 IN TXT "v=spf1 ip4:129.9.0.0/16" jeep.com. 3600 IN TXT "v=spf1ip4:129.9.75.0/24ip4:129.9.107.0/24include:smtp.fiatgroup.cominclude:mpsa .cominclude:spf.protection.outlook.cominclude:_spf.google.com"(Again, not technically a failure because the second record is garbage.)
dpreview.com. 300 IN TXT "spf2.0/pra include:amazon.com -all" "v=spf1 include:amazon.com -all"(Probably not what they intended — two records merged to a single non-SPF record.)
aa.org. 3600 IN TXT "\"v=spf1 include:spf.protection.outlook.com ip4:144.121.244.0/27 ip4:144.121.91.190 ip4:40.71.11.142 ip4:166.78.39.57 ip4:149.97.227.36/30 include:md02.com include:app.aventri.com -all\""(Oops, put quotes around the entire SPF record; what will the various SPF parsers do?)
SPF data is entered into DNS using various methods. It appears some DNS entry software allows or does extra quoting which is not covered by the RFC specification.
inquirer.net. 300 IN TXT "google-site-verification=4SuYiV92RkTRIUQmCD21iPZvs20hHGmAlOa_WcTXWpo\"\"v=spf1 include:_spf.google.com ~all\"\"facebook-domain-verification=xbz352h1e2ui4gbzp0k47la9v6vx2m"(Multiple TXT records merged to a single record plus some escaped quotes also seen in it.)
bumppy.com. 300 IN TXT "v=spf1 include:zcsend.net ~all\""(Trailing quote.)
debate.com.mx. 60 IN TXT "debate.com.mx. IN TXT \"v=spf1 mx a ip4:52.88.33.70/32 -all\""(Oops, a master zone format TXT record got embedded into a TXT record.) (See previous example too.)
Some mechanisms define an IPv4 or IPv6 address or network, but the operator or server-side software entered a non-IP value there.
zaobao.com. 60 IN TXT "sqtj6sq8im6ja91h0m0gpp85m1" "v=spf1 include: ip4:10/8 ip4:192.168/16 ip4:172.16/12 ip4:202.27.16.0/20 ip4:203.117.75.0/28 ip4:203.116.39.80/28 ip4:203.117.68.16/28 ~all"(While technically not a problem because junk at start of rrdata made it not an SPF record, the problem was with the 10/8, 192.168/16, and 172.16/12 networks as the RFC defines the IPv4 network starting with a dot-delimited IP address.)
natlawreview.com. 300 IN TXT "v=spf1 ip4:outbounds9.obsmtp.com ~all"(This domain also has multiple SPF TXT records failure so maybe this bogus ip4 value won't be even seen. The normal way to get an IP from a domain name is to use the "a" mechanism.)
Like above, an IP address is missing where the SPF record specifically has a mechanism key for it. If there is an IP address outside of the mechanism key, then that IP would be detected as an unknown mechanism too.
mockplus.cn. 600 IN TXT "v=spf1 a mx ip4: 121.40.151.193 -all"(Literal space between the "ip4:" mechanism and its value.)
myblog.de. 28800 IN TXT "v=spf1 mx a:mail.myblog.de ip4:164.68.105.106 ip6: 2a02:c207:3004:1232:0000:0000:0000:00011b:eff:fecd:c739 -all "(Again, but for "ip6:" mechanism.)
Other mechanisms or modifiers use a domain name as the value (which may be looked up), but the operator or server-side software entered a non-domain name there.
alphacoders.com. 300 IN TXT "v=spf1 include:u3333260.wl011.sendgrid.net include:168.245.114.232 -all include:icloud.com ~all"(The "include" mechanism needs a domain name to recurse to its TXT SPF record.)
Again, like the previous, a domain name is missing where the SPF record specifically has a modifier or mechanism indicating it.
bouncex.net. 300 IN TXT "v=spf1 include: _spf.google.com ~all"(Literal space after the "include:" mechanism.)
In the cases, where a IP address appears to define a IP network, the CIDR number is missing.
zdnet.com. 300 IN TXT "v=spf1 mx ip4:64.30.227.218 ip4:64.30.226.54/31 ip4:74.125.148.0/22 ip4:216.239.125.28/23 ip4:202.73.54.176/28 ip4:58.65.7.128/28 ip4:62.108.138.0/28 ip4:216.239.114.214/23 ip4:216.239.114.222/23include:_spf.google.com include:spf-00262c01.pphosted.com include:spf.protection.outlook.com -all"(The "include" mechanism got merged into a /23 CIDR.)
statnews.com. 300 IN TXT "v=spf1 a mx include:spf-00504401.pphosted.com ip4:50.203.72.0/24 ip4:66.151.183.0/24\013\010ip4:198.115.64.0/24 ip4:198.115.68.0/22 ip4:198.115.72.0/21\013\010ip4:198.115.66.0/23 ip4:198.115.80.0/21 ip4:198.115.65.0/24\013\010include:_spf.google.com ?all"(ASCII carriage return and newline got encoded into the rrdata, merged with the CIDRs.)
blr.com. 300 IN TXT "v=spf1 include:amazonses.com include:mailgun.org include:_netblocks.mimecast.com include:us-spf.email.litmos.com ip4:64.207.100.64 ip4:64.207.100.70 ip4:34.224.97.62/32 ip4:216.34.99.11/32 ip4:216.34.99.12/30 ip4:216.34.99.16/30; ip4:64.207.100.64/32; ip4:170.10.133.171/32 -all"(Semicolons aren't separators in SPF.)
Some domain names look like garbage or are not a domain name.
inner-active.mobi. 3600 IN TXT "v=spf1 mx mx:a include:secureserver.net ip4:23.20.171.99 -all"(Unsure what the "a" was meant for the domain name; maybe meant to have an "a" mechanism.)
votesmart.org. 1 IN TXT "v=spf1 a:secure.votesmart.org a:app0.votesmart.org ip4:35.165.188.123 include:v=spf1 include:_spf.google.com ~all include:v=spf1 include:sendgrid.net ~all ~all"("v=spf1" used as a domain name twice!)
SPF mechanisms delimit key from value using a colon.
discord.io. 300 IN TXT "v=spf1 ip4=78.46.32.25 include:discord.io -all"("ip4" mechanism has wrong delimiter.)
Again, the domain name is not a domain name.
eversource.com. 3600 IN TXT "v=spf1 mx include:_spf.kubra.com include:spf.mandrillapp.com include:u248959.wl210.sendgrid.net include:spf-0018e401.pphosted.com include:spf.protection.outlook.com include:mktomail.com include:mail.zendesk.com\" \" ip4:159.108.154.1 ip4:159.108.152.1 ip4:159.108.80.1 ip4:159.108.44.1 ip4:159.108.148.43 ip4:159.108.36.192 ip4:52.3.144.228 ip4:168.245.65.249 ip4:43.228.187.74 -all"(Simply caused by an escape quote; what will the receiving SPF implementation do?)
feed-xml.com. 300 IN TXT ""v=spf1 redirect=_spf.mailhostbox.com""(Something attempted to enter errant quotes, those got converted to HTML Entities and the checker saw that garbage with a few different checks.)
SPF modifiers delimit key from value using an equal sign.
truxgo.net. 38400 IN TXT "v=spf1 +ip4:131.196.253.94 +redirect:_spf.mailhostbox.com"("redirect" modifier has wrong delimiter.)
Qualifiers (like [-+~?]) are only used with SPF mechanisms.
truxgo.net. 38400 IN TXT "v=spf1 +ip4:131.196.253.94 +redirect:_spf.mailhostbox.com"("redirect" modifier doesn't use a "+" qualifier.)
So over a couple thousand identified SPF problems detected out of around 55 thousand domains. Our other SPF study found many other security problems related to these same SPF records including around 80 bad DNS names used for policy decisions.
DNS Institute's DNS auditing service checks for over 125 requirements or best practices, including for DNSSEC, EDNS, and IPv6, as defined by DNS vendors, IETF/RFCs, governments, and DNS registries.
Gab
Twitter