IntrusionOnline

Following the new “tradition” of posting a security tip of the week (mentioned here, here ; SANS jumped in as well), I decided to follow along and join the initiative. One of the bloggers called it “pay it forward” to the community.

So, Anton Logging Tip of the Day #15: Fear and Loathing in Event 567

This tip digs into a seemingly simple, but really VERY esoteric subject: monitoring file access and modification via a Windows event log. Now, some people - who never studied this subject - tend to have a very simplistic view of this: just enable Object Access auditing, then right-click on a file or directory, click Security->Advanced->Auditing and then pick what types of events will be logged and by what accessing entities (i.e. users or computers). OK, so this will produce some logs, that is for sure. But are they useful?

First, why are we doing this? We typically need to know the following when we audit file access in Windows (or any other OS for that matter) for security (monitoring and investigation) or compliance:

• Time/date
• Computer where it happened
• User who touched the file
• Application he used to access the file
• File name + location (directory, share, etc)
• Type of access (read, write, create, delete, etc)
• Status (i.e. success or failure)

Can we get this from the above logs? No.

What? No!?! Really?

Yes, really. We can get some of the above, some of the time, not all of the above, all of the time. Here is an example, we are looking at event ID 560 (picture) and then at an extract from its description field.

Event:

Description (selected field):

Object Server: Security

Object Type: File

Object Name: C:\0\TestBed\simple_text_file.txt

Image File Name: C:\WINDOWS\system32\notepad.exe

Primary User Name: Anton

Primary Domain: XXXXXX

Accesses: READ_CONTROL

SYNCHRONIZE

ReadData (or ListDirectory)

WriteData (or AddFile)

AppendData (or AddSubdirectory or CreatePipeInstance)

ReadEA

WriteEA

ReadAttributes

WriteAttributes

WTH is that? Well, we know that the user ‘Anton’ has successfully read? wrote? changed attributes? did something? with a file named “C:\0\TestBed\simple_text_file.txt” using a program named “C:\WINDOWS\system32\notepad.exe.” That’s the best we can get, in this case! We may try to look at event IDs 562 and 567, but this missing information (i.e. the exact action performed) will not be added.

BTW, there will be a few more dozen (sometime hundreds!) of the 560s, 562s and 567s produced - all from just opening the text file in a notepad. The above event is notable for having BOTH “notepad” and “simple_text_file.txt” in the same event; others will have either of the two.

Anything else gets in the way? Yes, lots! MS Office will write to all files, even just opened for reading (with no user modifications to the content whatsoever), which will screw up your log monitoring efforts. If the file is on a share, more information will be missing (e.g. username might be).

So, how to use Windows event logs for file access tracking?

1. Enable logging (as described above)
2. Pick events 560 (most useful) and 562, 567 (useful too)
3. Look for fun filenames that might be touched by the users (have a list of files and users handy)
4. Figure out what programs were used to access them (this is called “Image File Name” in “WinLogSpeak”)
5. Ponder the ‘Accesses’ section of each event until your brain turns blue or until you decide whether such access is authorized or not…

Overall, this is still very useful for file access monitoring, but the process is paaaaaainful.

BTW, I am tagging all the tips on my del.icio.us feed. Here is the link: All Security Tips of the Day.

Technorati tags: security, tips, logging, log management

My next fun logging poll is here - please vote! It is about tools for centralized collection of Windows Event Log from servers and other systems. One of the somewhat surprising discoveries from my previous poll was that few people look at Windows logs; this poll drills down into it.

And, don’t forget that ProjectLASSO Windows event collector allows people to grab Windows event logs remotely without those hated agents…

Past logging polls and their analysis:

Technorati tags: logs, logging, polls, log management

Following the tradition of posting a tip of the week (mentioned here, here ; SANS jumped in as well), I decided to follow along and join the initiative. One of the bloggers called it “pay it forward” to the community.

So, Anton Security Tip of the Day #14: More access_log Fun: What Are You Not GETting?

In this tip, we will look at some bizarre artifacts that show up in web server access logs today. Here we have a production log from an Apache web server that is full of interesting (and sometimes ominous!) little mysteries that we will investigate in order to determine their impact on security and operational health of the site.

Logs do contain more mysteries than we have time, so we will focus on a few of them: specifically, unusual web request methods. Let’s see who is trying to POST or use some other method (OPTIONS, HEAD, PUT or something - see a list here) on our site, instead of just GET’ting the content (GET command is used by web browsers to retrieve the pages, while POST is used to upload content, press buttons, etc - at least in “web 1.0″ land - see earlier tip #12 where POST request was found in proxy logs)

Here is one little artifact that attracted my attention due to a POST request vs a web forum as well as a battery of slashes (which actually increases in subsequent request - of which there were many)

10.10.102.250 - - [12/Feb/2008:16:10:50 -0500] “POST /phpBB3////ucp.php?mode=register&sid=e5efaa77a777066c61f71808e9e57b19 HTTP/1.0″ 200 14397 http://www.example.com/phpBB3///ucp.php?mode=confirm&id=7640df05c7e24b7acf7a68800fe6dc59&type=1&sid=e5efaa77a777066c61f71808e9e57b19 “Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.2) Gecko/20021126″

… more…

10.10.102.250 - - [12/Feb/2008:16:12:29 -0500] “POST /phpBB3///////////////ucp.php?mode=login&sid=e5efaa77a777066c61f71808e9e57b19 HTTP/1.0″ 200 9355 “http://www.example.com/phpBB3//////////////ucp.php?mode=login&sid=e5efaa77a777066c61f71808e9e57b19″ “Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.2) Gecko/20021126″

This one really is a mystery; what do we know about it? The server responded to the request OK (code 200), so the POST actually happened. The first request was a request to register with a web discussion board and the second was a request to login. Multiple slashes are actually ignored by the web server, so why put them in the request (no answer)? Also, I think that the User-Agent is spoofed … do you know why? Finally, if I see something like that in my logs, I will definitely investigate it, primarily due to the fact that Apache responded with 200 OK code.

The next one is so classic it it dumb (and so dumb, it’s a classic :-))

10.10.123.226 - - [12/Feb/2008:03:46:54 -0800] “POST /_vti_bin/shtml.exe/_vti_rpc HTTP/1.1″ 404 - “-” “MSFrontPage/6.0″

10.10.123.226 - - [12/Feb/2008:03:46:55 -0800] “OPTIONS / HTTP/1.1″ 200 20210 “-” “Microsoft Data Access Internet Publishing Provider Protocol Discovery”

It is probably one of the ancient IIS attacks (check out this fun BlackHat preso on that, circa 2003) - why would someone probe for it now is beyond me. In any case, Apache on Linux and “*.exe” don’t mix

The final log record is also fun:

10.10.101.222 - - [12/Feb/2008:15:33:22 -0800] “PUT /zk.txt HTTP/1.0″ 405 223 “-” “Microsoft Data Access Internet Publishing Provider DAV 1.1″

The above uses a PUT request which is pretty much deprecated now; the purpose of the above is clearly malicious. In fact, modern Apache shouldn’t even allow it, thus it responds with code 405 “Method Not Allowed.” Nothing to worry about (even though some poor critter got owned with that! BTW, if you follow that link, check out HTTP response code 201 - if you see it in your logs, run! :-))

Overall, if you see too many POSTs or too many “GET then POST” sequences from the same IP in rapid succession, investigate it since no legitimate access should produce such a pattern…

As further reading, I heartily recommend this paper: “Detecting Attacks on Web Applications from Log Files”

Also, I am tagging all the tips on my del.icio.us feed. Here is the link: All Security Tips of the Day.

Technorati tags: logging, log management, tips, security

As promised, here is another “Top 11 Reasons” which is about log analysis. Don’t just read your logs (definitely don’t just collect them); analyze them. Why? Here are the reasons:

1. Seen an obscure log message lately? Me too - in fact, everybody have. How do you know what it means (and logs usually do mean something) without analysis? At the very least, you might need to bring additional context to know what some logs mean (example: IP address -> hostname -> server owner)
2. Logs often measure in gigabytes and soon will in terabytes; log volume grows all the time - it definitely passed the limit of what a human can read a long time ago, it then made simple filtering ‘what logs to read’ impossible as well: automated log analysis is the only choice.
3. Do you peruse your logs in real time? This is simply absurd! However, automated real-time analysis is entirely possible (and some logs do crave for your attention ASAP - e.g. major system failures, confirmed intrusions, etc)
4. Can you read multiple logs at the same time? Yes, kind of, if you print them out on multiple pages to correlate (yes, I’ve seen this done :-)). Is this efficient? God, no! Correlation across logs of different types is one of the most useful approaches to log analysis.
5. A lot of insight hides in “sparse” logs, logs where a single record barely matters, but a large aggregate does (e.g. from one “connection allowed” firewall log to a scan pattern). Thus, the only way to extract that insight from a pool of data is through algorithms that “condense” that collection of logs into usable knowledge (some say, visualization is the way to go)
6. Ever did a manual log baselining? This is where you read the logs for a while and learn which ones are normal for your environment. Wonna do it again? Thought so Log baseline learning is a useful and simple log analysis technique, but humans can only do it for so much before burning out.
7. OK, let’s pick the important logs to review. Which ones are those? The right answer is “we don’t know, until we see them.” Thus, to even figure out which logs to read, you need automated analysis.
8. Log analysis for compliance? Why, yes! Compliance is NOT only about log storage (e.g. see PCI DSS). How to highlight compliance-relevant messages? How to see which messages will lead to a violation? How do you satisfy those “daily log review” requirements (again, see PCI DSS)? Through automated analysis, of course!
9. Logs allow you to profile your users, your data and your resources/assets. Really? Yes, really: such profiling can then tell you if those users behave in an unusual manner (in fact, the oldest log analysis systems worked like that). Such techniques may help reach the holy grail of log analysis: have the system automatically tell you what matters for you!
10. Ever tried to hire a log analysis expert? Those are few and far between. What if your junior analysts can suddenly analyze logs just as well? One log analysis system creator told me that his log data mining system enabled exactly that. Thus, saving a lot of money to his organization.
11. Finally, can you predict future with your logs? I hope so! Research on predictive analytics is ongoing, but you can only do it with automated analysis tools, not with just your head alone (no matter how big :-)) …

Past top 11 reasons:

• Top 11 Reasons to Collect and Preserve Computer Logs
• Top 11 Reasons to Look at Your Logs
• Top 11 Reasons to Secure and Protect Your Logs

Technorati tags: logging, log management, tips

This is my 6th logging poll (vote here now!)- links to the previous five polls below.

This one is deceptively similar to the #1 below, but it is not. This poll is What logs do you actually LOOK at? and not Which Logs Do You Collect? In other words, are you a log packrat? Are you collecting and never using the log data? You are making a mistake, if you don’t.

Past polls:

This poll is especially fun: What are your top challenges with logs and logging? Vote here.

Past polls were:

I just have to start with this quote from Rich Mogul: “… Legions of armchair futurists slobber over their keyboards, spilling obvious dribble that they either predict every year until it finally happens or is so nebulous that they claim success if a butterfly flaps its wings in Liechtenstein.” Amen to that, Rich. Onwards to my 2008 predictions!

So, just as in 2006 and 2007, I am coming up with security predictions that cover both technology and market. I just posted a review of my last’s year’s prediction where I mostly erred on the conservative side. I promise to be more ‘extreme’ this year, while still keeping the old wisdom of Richard Feynman in mind: if you predict the status quo, you are more likely to be correct…

Here is my ‘twitter-style’ (I guess what used to be called telegraph-style :-)) view of predictions in no particular order:

Platform security:

• Vista makes us secure = no. People start to actually use it (in large numbers) = maybe. And then get 0wned = yes! The volume of Vista hacking (and then Win 2008 hacking) will increase as the year progresses.
• Increase in Mac hacking = yes. The story is that Vista drives Mac adoption -> Mac increase in popularity will drive a new wave of Mac “0wnership”
• Web application hacking still on the growth path = yes. As they say, ‘it will get worse before it gets better.’ I am predicting that 2008 is still the year when it continues to be getting worse.

Vulnerabilities:

• 0days use becomes mundane = yes. This will be especially true for those browser-hacking folks who “need” to earn some cash off phishing and other data theft. Thus, “0day use” will no longer constitute news!

Hacking, data theft, etc:

• Loss of trust towards legitimate Internet sites = yes. This is manifested by things like this point by the WS guys - more 0wned than malicious sites are used to spread malware. Even now I shudder from the thought that ANY site I visit might be displaying a malicious banner ad which is either bought or “hacked in” by the attackers. The implications of this are pretty horrifying!
• Major utility/SCADA hack = no (not yet). Everybody predicts this one forever (as Rich mentions), but I am guessing we would need to wait another year or so for this …
• Cyber-terrorism = no (again, not yet!) Will it be a reality in the future? You bet! Just not now …
• A massive data theft to dwarf TJX = yes. And it will include not some silly credit card number (really, who cares? :-)), but full identity - SSN and all.

Malware:

• The year of mobile malware = no (not yet, if you insist!). As I discussed here, mobile malware is “a good idea” (for attackers) provided there is something valuable to steal (not the case yet in the US)
• More fun bots = yes. Bots are here to stay: they follow an overall trend for IT automation (seriously!). Think of bot infrastructures as “shadow IT” with their own SLAs, business model innovation, performance optimization tactics, etc
• Fewer worms and viruses = yes (why write one if you can make money off bots?) As the share of “conventional” viruses and worms in the whole malware universe decreases, so will the popularity of “legacy” AV vendors …
• Facebook malware/malicious app = yes . This one will be fun to see (others agree), and current malware defenses will definitely not stop this “bad boy.”On the flip side, there is not that much to steal off Facebook accounts …

Compliance:

• PCI DSS continues its march = yes. In fact, I bet PCI DSS frenzy will spread downmarket - there is sooooo much more Level 3s and Level 4s compared to Level 1 merchants. They all take CCs, they are all insecure - thus, they will all be 0wned! And then hopefully fined
• ISO17799, ITIL, COBIT frameworks = maybe (again); they likely won’t be ‘hot,’ at least not in the US; ad hoc approach (with some use of ideas from the above frameworks) to security management will still rule.

Risk management:

• Will we know what risk management actually is in the context of IT security = no. Some people (e.g here) might, but not the majority. And don’t even get me started on security ROI This part of security realm will continue to be occupied mostly by loudmouths who will spout, but never define; rant, but never explain; blab, but never clearly state. Sorry to those who are not like this, but you will continue to be in the minority in 2008.

Security technologies:

• eVoting security will flare up = yes. Expect big and bad stories about evoting in preparation to the US elections. Maybe another “chad story”, but with an “e-” added to it? Fun, fun, fun!
• Full disk encryption becomes popular = no. In fact, I predict that in 2008 encryption would be “the new firewall” - more and more people will hide from reality behind “we have encryption - we are safe now!” (check out my piece on encryption mistakes, while you are at it)
• NAC= huh. Huh? The451Group said it best: “NAC has been the ‘next big thing’ for about four years now – that’s a long time in the IT world.” Others just say “NAC fallout has started.” NAC vs insider attacks? Gimme a break…
• More whitelisting for host and network security = yes (but combined with blacklisting, which is certainly not going away!) As malware landscape becomes even more diverse, application whitelisting for security will start to shine even more.
• Academic security research stays ridiculous = yes. Wrong problems, wrong solutions, wrong speed (as in: solving solved problems of day before yesterday…). There will be some exceptions: for example, some of the Project Honeynet academic participants deliver a punch!
• Secure coding becomes mainstream = no (definitely, ‘not yet’ on this one) It pains me to say that that I think that while this ball definitely started rolling (e.g. SANS is pushing it hard now) it won’t be hurtling down the highway at full speed. 2009? Sure, may be!
• IPv6 = no (while most think ‘not yet’, some start thinking ‘not ever’) In other words, Internet ’secure by design’ = pipe dream in 2008.

Security market:

• Mid-market and SMB security = yes! I think 2008 is the year when smaller organizations will start buying the types of security solutions that were only looked at by the large enterprises before. After all, they have the same problems to solve! They have compliance too. They lose data
• More security SaaS (software as a service) = yes. It is not just Qualys anymore … More companies will figure out ways to sell security software as a service. This is especially true due to the SMB security spending increase predicted above!
• ‘Consolidation’ = no. Whaaaaat? You just said ‘no’ to consolidation in security market? Well, Vendor X might buy Vendor Z and Vendor N might go down in flames, but I predict that we will celebrate 2009 with just as many security vendors as we have today …

Logging and log management:

• Database logging = yes. 2008 is the year when database logs will be collected and analyzed just as Unix syslog, Windows event logs and firewall logs are collected and analyzed today by just about everybody.
• Application logging will start = yes. People will start collecting (at least collecting at first) application logs, not just firewall and server OS logs (and database logs, as mentioned above). Maybe ERP, CRM logs, maybe other large enterprise applications will lead the way. Major ‘application logging waterfall’ will occur later, however …
• Now that collection and management are ‘taken care of’ in many organizations, log analysis will (again…) come to the forefront = yes. In the end of 2008, we will be doing log analysis in a large number of fun, new ways - it won’t just be about rule-based correlation and keyword searching anymore (Andrew agrees)

Last year’s drag-ons and ongoing trends:

• Some things make dumb predictions since they are so pitifully obvious and have been going on for years already. Thus, I pile them in this section…
• So, client vs server exploitation: it started a few years back and will continue, for sure: more client vulnerabilities will be used to 0wn more desktops. Similarly, application vulnerabilities will beat platform ones. And targeted, commercially-driven attacks will overtake indiscriminate ones (another “no-brainer” that some try to sell as a prediction…)
• Both of the above will power further evolution of network and system security into data and broader information security (it will be happening for another 3-5 years)
• More fun “web 2.0″ threats will come our way, but then again, this is true about most of the technologies that are being actively adopted …

Dark horses, that will influence security in a major but unknown way in 2008:

• Virtualization = people talk about hypervisor security and virtual security appliances as well as other fun stuff (e.g. this), but, in all honesty, we can’t yet fathom the impact that the coming virtualization wave will have on information security.
• Privacy = I predict that privacy issues, also privacy laws and public outcry due to privacy violations will impact the world of information security in 2008. However, my crystal ball is refusing to share the details on how exactly, citing “privacy concerns”

Come back in Jan 2009 to see how I did!

Any comments? Additional predictions?

Technorati tags: security, future, predictions

Even though these posts are from my main blog ( see “Security Warrior” blog) and not from this one, the top posts would still be of interest to my readers here. So, enjoy!

These are my top popular “Security Warrior” blog posts for 2007! To make this a competition of posts, I am removing the links to the main blog, search labels (e.g. log management, which was indeed one of the most popular resources on the blog) as well as grouping posts together in theme clusters.

1. Same as during past few months, the “fallout” from being featured on a high-profile programming site continues to drive humongous loads of traffic which made this set of posts the most popular, even for the year. The topic that got such a huge boost was anti-virus efficiency. The posts are: Answer to My Antivirus Mystery Question and a “Fun” Story, More on Anti-virus and Anti-malware, Let’s Play a Fun Game Here … A Scary Game, The Original Anti-Virus Test Paper is Here!, Protected but Owned: My Little Investigation as well as a final entry about my own switch away from mainstream major-vendor anti-virus tool: A Bit More on AV and Closure (Kind of) to the Anti-Virus Efficiency/Effectiveness Saga.
2. Next by rank is a set of my Top11 lists: Top 11 Reasons to Collect and Preserve Computer Logs and Top 11 Reasons to Look at Your Logs (the third list, Top 11 Reasons to Secure and Protect Your Logs, was not quite that popular - I have long argued that, sadly, few people care about log security yet).
3. Wow! I love, love, love the fact that my blog readers made my first Common Event Expression (CEE), post introducing this emerging log standard, (official site now live!) one of the most popular: Finally, Common Event Expression (CEE) is Out!!!. My other CEE-related posts are labeled here.
4. Hurray to database logging (finally!) My posts related to database logging top the charts. Specifically, How to Do Database Logging/Monitoring “Right”? as well as its “prequels” Full Paper on Database Log Management Posted and On Database Logging and Auditing (Teaser + NOW Full Paper).
5. Finally, security ROI saga that flared up mid-year is also among the most popular. Indeed, Security ROI Pile-Up! post made it into Top5 (the related posts are: The Entire Security ROI Blood Trail and ROI, ROSI, RROI and Harry Potter Tales). The rest of my ROI-related posts are labeled here.
6. At the risk of destroying my math credibility, I will add an item #6 to my Top 5 list, again. This little post called On Open Source in SIEM and Log Management have also generated a lot of traffic and discussion. Indeed, log management vs SIEM as well as reasons for a lack of a popular and complete open source log management solution are fun topics!

See you in 2009!

Possibly related posts / past monthly popular blog round-ups:

• Monthly Blog Round-Up - December 2007
• Monthly Blog Round-Up - November 2007
• Monthly Blog Round-Up - October 2007
• Monthly Blog Round-Up - September 2007
• Monthly Blog Round-Up - August 2007

This is reposted here from my blog at http://www.securitywarrior.org!

Following the new “tradition” of posting a security tip of the week (mentioned here, here ; SANS jumped in as well), I decided to follow along and join the initiative. One of the bloggers called it “pay it forward” to the community.

So, Anton Security Tip of the Day #13: Into the Darkness … or The Ominous World of Unix Binary Audit Logs

In this tip, we will take a peek at one of the most esoteric areas of logging: Unix binary audit logs. Solaris BSM and Trusted Solaris auditing is the least unknown example of it, even though other Unix vendors have similar auditing capabilities - see this for HP-UX Audit and this for IBM AIX audit. Linux kernel audit is also pretty much the same thing. If you look for information on ‘Solaris BSM audit logs’ , you’d find plenty of tips on how to enable such logging, a little on how to manage/rotate the log files, a bit on how to survive the resulting data deluge and ALMOST NOTHING on what to do with the log data, which is kinda sad After looking at BSM logs for a while, I developed an opinion that nobody has ever looked at them on a regular basis

So, let’s assume you enabled Solaris BSM kernel audit for user “root” and few other “interesting” users (there is no per-object logging in Solaris; other Unix’es do have it) via the following commonly recommended per-user configuration in /etc/security/audit_user:

root:lo,ad,fw:no

anton:all,-all:no

jsmith:all,-all:no

This config pretty much records all the actions by the users listed. Now, you have audit files growing like shrooms in you /var/audit. What good does it give us? First, we need to convert the binary audit files into text - something along the lines of

# auditreduce -A /var/audit/20071127193515.not_terminated.SunUltra10 | praudit -l > /tmp/sol_box_11272007

will do. Now what? In this tip we will learn how use the audit logs to see who is trying to copy sensitive files off the system.

First, who is connecting out - lets’s search the logs for ‘connect’ calls (if you are using LogLogic for it, use Index Search for this task; if not, grep will have to do, but be prepared to wait). A few recommended searches:

• “connect AND 172.16.10.*” or “connect AND NOT 172.16.10.*” (to look for connection to specific IPs or to the outside networks) or simply ‘connect AND username’

Here is an example found (with connect, IP and user in bold):

header,103,2,connect(2),,Tue Nov 27 11:36:46 PST 2007, + 193 msec,argument,1,0×4,so,socket,0×0002,0×0002,0×80d6,SunUltra10,0×0016,10.1.1.41,subject,root,anton,other,anton,other,29902,29720,0 1611 172.16.0.173,return,success,0

At this point we already know the user name of the user who run that connecting process since it will be in the results (you can also the user to search as I showed above).

Next, what are those connections - let’s try to uncover which programs actually connected (BSM logs don’t make that easy). Let’s search for process starts in the same time frame:

• “execve AND NOT ls AND NOT <whatever other commands you don’t care to see>” will give you a list of started programs.

Example:

header,124,2,execve(2),,Tue Nov 27 11:36:46 PST 2007, + 115 msec,path,/usr/bin/scp,attribute,100555,root,bin,136,1573,0,subject,root,anton,other,anton,other,29901,29720,0 1611 172.16.0.173,return,success,0

Notice that both records have the same timestamps. Sadly, time and parent process ID ( which is in our case 29720) is all that correlates them together.

Finally, what file was affected (i.e. copied off the system via scp in this case) - more digging is in order; we again use the process ID and time. The easiest is to search for a file name or browse all records around the same time frame (might be A LOT!):

• “*secret.zip* AND anton” will work; we can add the above process ID and look for “anton AND 29720″ (but expect a lot of data since this is a shell process ID)

For example:

header,135,2,open(2) - read,,Tue Nov 27 11:36:47 PST 2007, + 900 msec,path,/tmp/not-so-secret.zip.gz,attribute,100600,anton,other,0,32743959,18446744073709551615,subject,root,anton,other,anton,other,29901,29720,0 1611 172.16.0.173,return,success,4

What do we know now? This user connected to this system and MAYBE copied this file via, MAYBE via scp. How cool is that? (A: not cool at all, since we are not sure!)

To conclude, if you can avoid dealing with Solaris BSM logs, please do so On a more serious note, you now know why these logs were called “the ugliest logs ever.”

Even more seriously (but still pretty humorously), these logs are a classic example of trees that make every effort to obscure the forest, because they record syscalls and not processes or user actions (and connect, execve and read are all logged separately). There are also many, many more idiosyncrasies (and, in fact, idiocies) where these come from

Also, I am tagging all the tips on my del.icio.us feed. Here is the link: All Security Tips of the Day.

The idea came from Jeremiah Grossman (here) when he described “The Best Web Application Vulnerability Scanner in the World” thus: “Within a few moments of pressing the scan button it’ll find every vulnerability, with zero false positives, generate a pretty looking report, and voila you’re compliant with GLBA, HIPAA, and PCI-DSS. Of course, we all know such a web application scanner is simply not possible to create for a variety of reasons.”

So, let’s imagine the idea log management application.

1. Logging configuration: the ideal log app will go and find all possible log sources (systems, devices, applications, etc) and then enable the right kind of logging on them according to a high level policy given to it (required: God-like powers)
2. Log collection: it will collect all the above logs securely (and without using any risky super-user access ) and with little to no impact to networks and systems (required: God-like powers)
3. Log storage: it can security store the above logs in the original format for as long as needed and in a manner allowing quick access to them - in both raw and summarized/enriched form (required: plenty of hardware)
4. Log analysis: this ideal application will be able to look at all kinds of logs, known to it and previously unseen, from standard and custom log sources, and tell the user what they need to know about their environment and based on their needs: what is broken? what is hacked? where? what is in violation of regulations/policies? what will break soon? who is doing this stuff? The analysis will power all of the following: automated actions, real-time notifications, long-term historical analysis as well as compliance relevance analysis (required: AI)
5. Information presentation: this tool will distill the above data, information and conclusions generated by the analytic components and present then in a manner consistent with the user’s role: from operator to analyst to engineer to executive. Interactive visual and drillable text-based data presentation across all log sources. The users can also customize the data presentation based on their wishes and job needs, as well as information perception styles (required: nothing more than a bunch of daring UI designers)
6. Automation: the ideal log management tool will be able to take limited automated actions to resolve discovered and confirmed issues as well as generate guidance to users so that they know what actions to take, when full-auto mode is not appropriate. The responses will range from full-auto actions to assisted actions (’click here to fix it’) to issuing detailed remediation guidance. The output will include a TODO-list of discovered items complete with actions suggested, ordered by priority (required: AI + some luck + some user stupidity :-))
7. Compliance: this tool can also be used directly by auditors to validate or prove compliance with relevant regulations by using regulation-specific content and all the collected data. The tool will also point at gaps in data collection as it applies to specific regulations that the user is interested in complying (required: God-like powers)

In other words, this magic black box will have crap shoveled from one side and will have answers to questions about the meaning of Life coming out the other side…

What? Am I nuts? Well, can I dream for a second?