MozillaWiki

Security/Mentorships/MWoS/2014/Linux Audit heka plugin (Go): Difference between revisions

Page Discussion

Security/Mentorships/MWoS/2014/Linux Audit heka plugin (Go) (view source)

Revision as of 19:18, 20 February 2015

6,091 bytes added ,  20 February 2015
no edit summary
No edit summary
 
(11 intermediate revisions by 2 users not shown)
Line 8: Line 8:
* Hardik Juneja
* Hardik Juneja
* Arun Sori
* Arun Sori
* Aalekh Nigam
* Aalekh Nigam [https://wiki.mozilla.org/User:Aalekhn aalekhn]
* Professor: Sanjay Goel
* Professor: Sanjay Goel  
* Mozilla Advisor: [https://mozillians.org/en-US/u/kang/ Guillaume Destuynder]
* Mozilla Advisor: [https://mozillians.org/en-US/u/kang/ Guillaume Destuynder]


== Project ==
== Project ==
Heka is a Mozilla project for logs routing, analysis, etc. (see http://hekad.readthedocs.org/en/latest/). Linux Audit logs are collecting various system calls and events in order to send them to a C user space program (auditd) over the netlink protocol. A Mozilla C plugin (https://github.com/gdestuynder/audisp-cef and https://github.com/gdestuynder/audisp-json) currently correlate, transforms, and send these events back to our logging architecture.
Heka is a Mozilla project for logs routing, analysis, etc. (see http://hekad.readthedocs.org/en/latest/). Linux Audit logs are collecting various system calls and events in order to send them to a C user space program (auditd) over the netlink protocol. A Mozilla C plugin (https://github.com/gdestuynder/audisp-cef and https://github.com/gdestuynder/audisp-json) currently correlate, transforms, and send these events back to our logging architecture.
See also https://air.mozilla.org/mozilla-winter-of-security-audit-go-project/ for a complete presentation of the project.


=== Description ===
=== Description ===
Line 19: Line 21:


This means it will listen for events from the kernel via the Netlink protocol, parse the messages, convert them (to JSON using MozDef's native format), and pass them over to Heka.
This means it will listen for events from the kernel via the Netlink protocol, parse the messages, convert them (to JSON using MozDef's native format), and pass them over to Heka.
[https://github.com/mozilla/audit-go Project Repository]


=== Success Criteria ===
=== Success Criteria ===
Line 25: Line 29:
=== Roadmap ===
=== Roadmap ===


Todo - add deadlines to this roadmap
* Get more familiar with Go [August 18 2014]
* Get more familiar with Go
* Work on communication using Netlink (receive audit messages from the kernel) [September 6 2014]
* Work on communication using Netlink (receive audit messages from the kernel)
* Port the netlink code as Heka Plugin and start working with lua. [September 17 2014]
* Store messages in some structure/process them (lua)
* Store messages in some structure/process them (lua) [September 24 2014]
* Correlate the kernel messages into a single mozdef-json message
* Correlate the kernel messages into a single mozdef-json message [September 31 2014]
* send the message back to MozDef (can be done through Heka functions) over HTTPS
* Send the message back to MozDef (can be done through Heka functions) over HTTPS [October 13 2014]


== Updates ==
== Updates ==
Line 42: Line 46:
* Get familiar with GO (first week)
* Get familiar with GO (first week)
* explore Netlink protocol, play with it and see how we can use it.
* explore Netlink protocol, play with it and see how we can use it.
=== 2014-08-25 ===
[https://etherpad.mozilla.org/YnR9hhqutn (Etherpad Index)]
[https://github.com/AALEKH/Audit-Go/ Project Code] 
Task for this week:
* Write code that uses Netlink protocol.
* Receive Audit Messages from kernel.
=== 2014-09-03 ===
[https://etherpad.mozilla.org/YnR9hhqutn (Etherpad Index)]
[https://github.com/AALEKH/Audit-Go/ Project Code] 
Current Work:
* Wrote Netlink code for recieving audit messages (Code present in testing branch of the repository).
* Recieved invalid argument error while running the code.
* Problem probably due Header type argument.
Discussion Point:
* Cause of invalid Argument
* Debugging techniques in linux
* Problem in implementing strace
Upcoming Work:
* Take refererence from libaudit.h and insert some other proto decimal code
* Use strace tool to got through netlink execution
** Use sudo -s and then strace -f go run <filename>
* Study strace and ltrace documentation
=== 2014-09-08 ===
[https://etherpad.mozilla.org/YnR9hhqutn (Etherpad Index)]
[https://github.com/AALEKH/Audit-Go/ Project Code] 
Current Work:
* Invalid Argument error removed :D, yay .
* Strace'd of auditd and netlink retrieved.
* Recieving nil as an output to audit message.
Discussion Point:
* Cause of nil error
* Way to parse audit message.
* Data type of audit message
Upcoming Work:
* Change the status code of proto, currently assigned 1000 status code is not correct.
* Output form of netlink message is structure.
* Use structure format same as audit_reply present in here [https://fedorahosted.org/audit/browser/trunk/lib/libaudit.h libaudit.h]
* Make a parse function like audit_get_reply as present in [https://fedorahosted.org/audit/browser/trunk/lib/libaudit.h libaudit.h]
* Study audit documentation and [https://fedorahosted.org/audit/browser/trunk/lib/libaudit.h libaudit.h] code
=== 2014-09-15 ===
Current Work:
* Added AuditRuleData Structure
* Added a AuditParse Function
Discussion Point:
* Cause of improper parsing.
* Follow the flow of auditd daemon from reading the code.
Upcoming Work:
* Structuring code to follow appropriate steps to enable Audit and receive response.
* Proper Parsing of a netlink response.
=== 2014-09-22 ===
* University Exams from 22 Sep to 26 Sep.
* Work will be resumed from 26 Sep.
=== 2014-09-27 ===
Current Work:
* Added AuditStatus Structure.
* Added a AuditSet Function.
* Appended Byte stream in netlinkMessage.
Discussion Point:
* Proper parsing of AuditStatus struct.
* Enabling Audit from our program.
Upcoming Work:
* Working AuditSet function
* Successfully Enabling Audit.
* Added AuditIsEnable Function.
* Moving current code to a go package.
=== 2014-10-04 ===
Current Work:
* AuditSet() Response successfully parsed into a AuditStatus struct.
* AuditIsEnabled Added.
* A netlinkAudit package.
Discussion Point:
* Adding Rules to audit.
* Receiving further responses from kernel.
Upcoming Work:
* Cleaning up old work; A simple driver program.
* Adding Rules to the Audit System.
* Adding Syscall information in a Rule struct.
=== 2014-10-13 ===
Current Work:
* AuditAddRuleData() Added.
* Adding rules to Audit kernel is successful.
Discussion Point:
* Remove hardcoded code for adding Rules.
* A JSON file like audit.rules.
* A way to map syscalls.
Upcoming Work:
* Moving Further ahead to a fully follow the auditd code flow.
* Adding AuditSetPid for registering our code with kernel.
* Adding Rules from a JSON File.
* Concurrent Way to run the code (Adding Channels).
=== 2014-10-20 ===
Current Work:
* X86 json added for mapping syscalls to a integers
* Changing design of the code.
* A new branch for work on the configuration part.
Discussion Point:
* Receiving response for the rules added
* Parsing the kernel response successfully.
Upcoming Work:
* A Getreply Function.
* Successful additon of rule from audit.rules.json file.
* Successful detection of the log message types and appropriate action.
* Diwali Holidays! More code on the way.
=== 2014-10-28 ===
Current Work:
* Successfully Parsing the responses received from kernel !!(Yay).
* Better Design; AutoGeneration of sequence numbers.
* More constants added.
* Mistakes in channel version. A simple version is added.
* Syscall mapping perfected.
* DeleteAllRules function added.
* Loading one or more rules from audit.rules.json.
Discussion Point:
* Addition of fields on the rules.
* Channel version of the getreply function.
Upcoming Work:
* A concurrent Getreply Function.
* A field mapping json file.
* More design changes.
=== 2014-11-4 ===
Current Work:
* A field map in JSON.
* Adding logs in a file.
* Channel Version added.
* Simplified Design.
Discussion Point:
* Working Field Addition and Error Handling.
* Porting code to work as Heka Plugin.
* Input type ( Http/Tcp/Udp ) for the Heka plugin.
Upcoming Work:
* A Final Field Version
* Prettify the output Messages.
* Moving on to the Heka Part.
=== 2014-11-28 ===
Current Work:
* A Final Field Version
* Prettify the output Messages.
* Moving on to the Heka Part.
* blocking points
Discussion Point:
* Creating Lpeg Grammer
* Audit-Heka Plugin discussion
* Use of LUa Decoder and Encoder
 
Upcoming Work:
* Lpeg Grammer
* Sample Decoder.
=== 2014-12-03 ===
Current Work:
* Lpeg Grammer
* Sample Decoder.
Discussion Point:
* Decoder Type
* Encoder Type
 
Upcoming Work:
* getPass for Golang
* CleanUp Golang Code
* Reconstruct Grammer
=== 2014-12-15 ===
Current Work:
* Lpeg Grammer
* Decoder Complete
Discussion Point:
* Decoder Errors
 
Upcoming Work:
* Cleanup Code
* Push to Mozilla Repository
=== 2015-02-20 ===
Presentation of the project available on AirMozilla https://air.mozilla.org/mozilla-winter-of-security-audit-go-project/


=== <date> ===
=== <date> ===
Gdestuynder
Confirmed users
502

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1006027...1057255"