I like the tool !!!
The Cpanel Modsecurity plugin shows log entries but the cmc seems to show the log is empty or nearly so
seems that the interfaces are looking in diiferent locations for logs
this may be expected behavior
Im no modsecurity guru
Where can I look to be sure that the log links are looking at the same logs ?
log access
-
ForumAdmin
- Moderator
- Posts: 1523
- Joined: 01 Oct 2008, 09:24
From the cmc help:
Note: The cPanel installation of mod_security includes an hourly cron job (/etc/cron.hourly/modsecparse.pl) which processes the mod_security log and places the entries into a mysql database for use with the WHM ModSecurity UI addon. That process empties out the log file, so this utility can only display those mod_security log entries that have been logged since the last time that cron job ran. There is an option on the main page to disable this cron job to allow this utility to view all the available log entries. You should use logrotate to rotate this along with the other apache logs on a regular basis.
-
silver_2000
- Junior Member
- Posts: 115
- Joined: 18 Dec 2006, 01:55
thanks for the heads up about the timing
Makes more sense now - should have RTFM
This morning the logs still dont seem to match up - I site that has no intreractive pieces is listed in the mod sec log from this tool but those same kinds of entries are not in the other log ....
Just trying to get a feel for what to expect
Makes more sense now - should have RTFM
This morning the logs still dont seem to match up - I site that has no intreractive pieces is listed in the mod sec log from this tool but those same kinds of entries are not in the other log ....
Just trying to get a feel for what to expect
Morning...
I'm resurrecting this thread because I'm seeing a completely empty log via the WHM -> CMC > ModSecurity Log button.
But when I actually check the path specified on the page (ie: /usr/local/apache/logs/modsec_audit.log ) the file is positively brimming with entries.
The WHM > Mod_Security log is likewise not updating.
[EDIT: Realised the log should only be readable before it's parsed (see next post) - Have left this post as is, as it'll probably be found by others confused about what is/should be happening]
I've just found that the modsecparse.pl was disabled so I'm guessing this is the root cause and I've since enabled it. I'd appreciate a confirmation that this is likely to be the cause of the lack of data though.
If so, a couple of suggestions to add:
1. If the log view in MMC requires the data to have been pulled across to the DB it would make sense to run the perl script to parse any new entries before displaying the log otherwise any debugging becomes anywhere up to 59 minutes out of date
2. Again, if the log view requires the data to be parsed, then it should check the status of the perl script and display a warning that the data is not being parsed hence the lack of data.
Obviously I'm basing these suggestions on an assumption but if you could correct or comment that would be appreciated.
Cheers
I'm resurrecting this thread because I'm seeing a completely empty log via the WHM -> CMC > ModSecurity Log button.
But when I actually check the path specified on the page (ie: /usr/local/apache/logs/modsec_audit.log ) the file is positively brimming with entries.
The WHM > Mod_Security log is likewise not updating.
[EDIT: Realised the log should only be readable before it's parsed (see next post) - Have left this post as is, as it'll probably be found by others confused about what is/should be happening]
I've just found that the modsecparse.pl was disabled so I'm guessing this is the root cause and I've since enabled it. I'd appreciate a confirmation that this is likely to be the cause of the lack of data though.
If so, a couple of suggestions to add:
1. If the log view in MMC requires the data to have been pulled across to the DB it would make sense to run the perl script to parse any new entries before displaying the log otherwise any debugging becomes anywhere up to 59 minutes out of date
2. Again, if the log view requires the data to be parsed, then it should check the status of the perl script and display a warning that the data is not being parsed hence the lack of data.
Obviously I'm basing these suggestions on an assumption but if you could correct or comment that would be appreciated.
Cheers
Just to add...
I've run the script manually and it's cleared the log but now the information is unavailable to view via the log button... I've confirmed that the information has made it into the database, that is apparently created, but aside from phpmyadmin I can't access it any other way
EDIT: On re-reading the thread this seems to be "by design"... So to recap it seems that:
1. The CMC is failing to read the modsec_audit.log at all (perms are 644 for that file)
2. The information stored in the database is actually not read at all and no interface has been provided for that purpose so phpmyadmin is the required method...
NB: For anyone looking for it... Look for the modsec database in phpmyadmin (you'll need to be using phpmyadmin as root) and you'll find the information in the modsec table.
In terms of a useful feature it would doubtless be valuable to have some kind of GUI to display the database logged errors and some means of searching on the basis of domain, rule, etc... for useful debugging. I appreciate the CMC is free but if you're wondering that would be an obvious port of call..
Cheers.
I've run the script manually and it's cleared the log but now the information is unavailable to view via the log button... I've confirmed that the information has made it into the database, that is apparently created, but aside from phpmyadmin I can't access it any other way
EDIT: On re-reading the thread this seems to be "by design"... So to recap it seems that:
1. The CMC is failing to read the modsec_audit.log at all (perms are 644 for that file)
2. The information stored in the database is actually not read at all and no interface has been provided for that purpose so phpmyadmin is the required method...
NB: For anyone looking for it... Look for the modsec database in phpmyadmin (you'll need to be using phpmyadmin as root) and you'll find the information in the modsec table.
In terms of a useful feature it would doubtless be valuable to have some kind of GUI to display the database logged errors and some means of searching on the basis of domain, rule, etc... for useful debugging. I appreciate the CMC is free but if you're wondering that would be an obvious port of call..
Cheers.
Did a bit more digging and debugging and it seems the problem may simply be down to the way the log is recording the data...
The code above seems to parse out the log to grab what it needs and then display it... if it gets nothing in the @requests array it displays the error...
Now I've confirmed it's pulling the lines out with a simple modification to print the $line at the beginning of the while (my $line = <IN>) loop but for whatever reason it's not parsing out properly.
So, to aid with debugging for anyone who can answer this question here's a sample of the modsec log:
Naturally I've replaced the domain info but otherwise that's as pulled from the log...
Any takers?
Code: Select all
while (my $line = <IN>) {
chomp $line;
if ($line =~ /^\=\=(\w*)\=*$/) {
$start = $1;
$entry = "";
}
elsif ($line =~ /^\-\-(\w*)\-A\-\-$/) {
$start = $1;
$entry = "";
}
elsif ($line =~ /^\-\-$start\-\-$/ and $start) {
push @requests, $entry;
$start = 0;
$entry = "";
}
elsif ($line =~ /^\-\-$start-Z\-\-$/ and $start) {
push @requests, $entry;
$start = 0;
$entry = "";
}
elsif ($start) {
$entry .= "$line\n";
}
}
close (IN);Now I've confirmed it's pulling the lines out with a simple modification to print the $line at the beginning of the while (my $line = <IN>) loop but for whatever reason it's not parsing out properly.
So, to aid with debugging for anyone who can answer this question here's a sample of the modsec log:
Code: Select all
www.foo.co.uk 80.229.82.161 - - [08/Feb/2010:12:01:57 +0000] "GET /product_images/ HTTP/1.1" 404 17416 "-" "-" S2-9NVVcVyAAAEBK@v4AAAAC "-" /20100208/20100208-1201/20100208-120157-S2-9NVVcVyAAAEBK@v4AAAAC 0 1221 md5:a391b7c2f3335329a4ac51f122f2fe52
www.foo.co.uk 80.229.82.161 - - [08/Feb/2010:12:02:00 +0000] "GET /product_images/ HTTP/1.1" 404 17416 "-" "-" S2-9OFVcVyAAAD320HUAAAAK "-" /20100208/20100208-1202/20100208-120200-S2-9OFVcVyAAAD320HUAAAAK 0 1412 md5:f633dec9e4d0b12102f390cbd2e65bce
www.foo.co.uk 78.133.51.234 - - [08/Feb/2010:12:04:21 +0000] "GET /product_images/ HTTP/1.1" 404 17416 "-" "-" S2-9xVVcVyAAAETRF8oAAAAE "-" /20100208/20100208-1204/20100208-120421-S2-9xVVcVyAAAETRF8oAAAAE 0 1600 md5:12aaf3df0fe7912207b202a3bd0270dc
www.footoo.org.uk 80.68.80.222 - - [08/Feb/2010:12:07:16 +0000] "GET /forum/rss.php?f=3&c=5&login HTTP/1.0" 401 24 "-" "-" S2-@dFVcVyAAAETyJqQAAAAA "-" /20100208/20100208-1207/20100208-120716-S2-@dFVcVyAAAETyJqQAAAAA 0 711 md5:36a11db3049f33e029b005f9c29e43ce
Any takers?
Ok... now pretty sure I know what is going on...
Typing out the previous post I now realise that the parsing routine is looking for a specific format that is provided as the default for CPanels mod_security implimentation.
I have the Atomic rules installed complete with the modification to, yep you guessed it, the log format, specifically the configuration of
Soooo, with this in mind I now know what's going on... BUT it would be nice to get this working so would it be possible to have a regex for the code in the preceding post that will work with the Atomic rules configuration... That's a big "please" added to that 
Cheers
Typing out the previous post I now realise that the parsing routine is looking for a specific format that is provided as the default for CPanels mod_security implimentation.
I have the Atomic rules installed complete with the modification to, yep you guessed it, the log format, specifically the configuration of
Code: Select all
SecAuditLogParts ABIFHZCheers
You have to check what you have written in your MODSEC2.CONF and MODSEC2.USER.CONF files as there is the error.
Check that MODSEC2.CONF are set as follow:
This is how MODSEC2.USER.CONF has to be set to use CMC and ATOMICORP rules:
If you do this, you will not have any issues with your rules and CMC.
Sergio.
Check that MODSEC2.CONF are set as follow:
Code: Select all
LoadFile /opt/xml2/lib/libxml2.so
LoadFile /opt/lua/lib/liblua.so
LoadModule security2_module modules/mod_security2.so
<IfModule mod_security2.c>
SecRuleEngine On
# See http://www.modsecurity.org/documentation/ModSecurity-Migration-Matrix.pdf
# "Add the rules that will do exactly the same as the directives"
# SecFilterCheckURLEncoding On
# SecFilterForceByteRange 0 255
SecAuditEngine RelevantOnly
SecAuditLog logs/modsec_audit.log
SecDebugLog logs/modsec_debug_log
SecDebugLogLevel 0
SecDefaultAction "phase:2,deny,log,status:406"
SecRule REMOTE_ADDR "^127.0.0.1$" nolog,allow
Include "/usr/local/apache/conf/modsec2.user.conf"
</IfModule>
Code: Select all
SecRequestBodyAccess On
SecResponseBodyAccess On
SecResponseBodyMimeType (null) text/html text/plain text/xml
SecResponseBodyLimit 2621440
SecServerSignature Apache
SecUploadDir /var/asl/data/suspicious
SecUploadKeepFiles Off
SecAuditLogParts ABIFHZ
SecArgumentSeparator "&"
SecCookieFormat 0
SecRequestBodyInMemoryLimit 131072
SecDataDir /var/asl/data/msa
SecTmpDir /tmp
SecAuditLogStorageDir /var/asl/data/audit
SecResponseBodyLimitAction ProcessPartial
SecDataDir /var/asl/data/msa
# ConfigServer ModSecurity whitelist file
Include /usr/local/apache/conf/modsec2.whitelist.conf
#ASL Rules
Include /usr/local/apache/conf/modsec_rules/*asl*.conf
Sergio.
Re: log access
Hello,
if you use ConfigServer ModSecurity Control, setting
SecAuditLogType Concurrent
doesn’t show logs in ConfigServer ModSecurity Log
the corrects setting in
modsec2.user.conf
is
SecAuditLogType Serial
if you use ConfigServer ModSecurity Control, setting
SecAuditLogType Concurrent
doesn’t show logs in ConfigServer ModSecurity Log
the corrects setting in
modsec2.user.conf
is
SecAuditLogType Serial