<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>iWebDev&#039;s blog &#187; database</title>
	<atom:link href="http://www.iwebdev.it/blog/?feed=rss2&#038;tag=database" rel="self" type="application/rss+xml" />
	<link>http://www.iwebdev.it/blog</link>
	<description>and everything goes around</description>
	<lastBuildDate>Sat, 06 Aug 2011 23:08:45 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.2</generator>
		<item>
		<title>Collect syslog events to database (second part)</title>
		<link>http://www.iwebdev.it/blog/?p=163</link>
		<comments>http://www.iwebdev.it/blog/?p=163#comments</comments>
		<pubDate>Thu, 10 Mar 2011 22:31:36 +0000</pubDate>
		<dc:creator>to</dc:creator>
				<category><![CDATA[Log event]]></category>
		<category><![CDATA[Unix]]></category>
		<category><![CDATA[database]]></category>
		<category><![CDATA[destination]]></category>
		<category><![CDATA[log]]></category>
		<category><![CDATA[mysql]]></category>
		<category><![CDATA[postgres]]></category>
		<category><![CDATA[source]]></category>
		<category><![CDATA[syslog-ng]]></category>
		<category><![CDATA[syslog-ng.conf]]></category>

		<guid isPermaLink="false">http://www.iwebdev.it/blog/?p=163</guid>
		<description><![CDATA[In the previous post you installed the syslog-ng 3.2.2. Now you have to configure our syslog-ng daemon to collect events to database; for this tutorial we choosed a MySQL and Postgres databases. First of all you have to configure the syslog-ng configuration file. nano /opt/syslog-ng/etc/syslog-ng.conf Syslog-ng receives log messages from a source. To define a [...]]]></description>
			<content:encoded><![CDATA[<p>In the previous post you installed the syslog-ng 3.2.2. Now you have to <strong>configure our syslog-ng daemon to collect events to database</strong>; for this tutorial we choosed a <strong>MySQL</strong> and <strong>Postgres</strong> databases. First of all you have to configure the syslog-ng configuration file.</p>
<blockquote><p>nano /opt/syslog-ng/etc/syslog-ng.conf</p></blockquote>
<p>Syslog-ng receives log messages from a <strong>source</strong>. To define a source you should follow the following syntax:</p>
<blockquote><p>source &lt;identifier&gt; { source-driver(params); source-driver(params); &#8230; };</p></blockquote>
<p>For example you have to define the following source:</p>
<blockquote>
<div>source my_source{ tcp ( port ( 614 ) ); };</div>
</blockquote>
<p>In syslog-ng log messages are sent to files. The <strong>destination</strong> syntax is very similar to sources:</p>
<blockquote><p>destination &lt;identifier&gt; {destination-driver(params); destination-driver(params); &#8230; };</p></blockquote>
<p>You will be normally logging to a file, but you could log to a different destination-driver: pipe, unix socket, TCP-UDP ports, terminals or to specific programs.</p>
<blockquote>
<div>destination my_dest{ file(&#8220;/var/log/mylog.txt&#8221;); };</div>
</blockquote>
<div>If you want to collect syslog to database you have to create mysql database and table</div>
<blockquote><p>CREATE DATABASE `syslog` DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci;</p>
<p>USE `syslog`;</p>
<p>CREATE TABLE IF NOT EXISTS `logs` (<br />
`id` bigint(20) unsigned NOT NULL auto_increment,<br />
`host` varchar(128) collate utf8_unicode_ci default NULL,<br />
`facility` varchar(10) collate utf8_unicode_ci default NULL,<br />
`priority` varchar(10) collate utf8_unicode_ci default NULL,<br />
`level` varchar(10) collate utf8_unicode_ci default NULL,<br />
`tag` varchar(10) collate utf8_unicode_ci default NULL,<br />
`datetime` datetime default NULL,<br />
`program` varchar(15) collate utf8_unicode_ci default NULL,<br />
`msg` text collate utf8_unicode_ci,<br />
`seq` bigint(20) unsigned NOT NULL default &#8217;0&#8242;,<br />
`counter` int(11) NOT NULL default &#8217;1&#8242;,<br />
`fo` datetime default NULL,<br />
`lo` datetime default NULL,<br />
PRIMARY KEY  (`id`),<br />
KEY `datetime` (`datetime`),<br />
KEY `sequence` (`seq`),<br />
KEY `priority` (`priority`),<br />
KEY `facility` (`facility`),<br />
KEY `program` (`program`),<br />
KEY `host` (`host`) )<br />
ENGINE=MyISAM  DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;</p>
<p>GRANT SELECT , INSERT , UPDATE , DELETE , CREATE , DROP , INDEX , ALTER ON `syslog` . * TO &#8216;syslog&#8217;@'localhost&#8217;;</p>
<p>SET PASSWORD FOR &#8216;syslog&#8217;@'localhost&#8217; = PASSWORD( &#8216;syslog&#8217; )</p></blockquote>
<p>Edit syslog-ng config appropriately; add these rows in the destination section (if you want to use Postgres you have to change mysql to pgsql):</p>
<blockquote><p>sql(type(mysql)<br />
host(&#8220;localhost&#8221;)<br />
username(&#8220;syslog&#8221;)<br />
password(&#8220;syslog&#8221;)<br />
database(&#8220;syslog&#8221;)<br />
table(&#8220;logs&#8221;)<br />
columns(&#8220;host&#8221;, &#8220;facility&#8221;, &#8220;priority&#8221;, &#8220;level&#8221;, &#8220;tag&#8221;, &#8220;datetime&#8221;, &#8220;program&#8221;, &#8220;msg&#8221;, &#8220;seq&#8221;)<br />
values(&#8220;$HOST_FROM&#8221;, &#8220;$FACILITY&#8221;, &#8220;$PRIORITY&#8221;, &#8220;$LEVEL&#8221;, &#8220;$TAG&#8221;, &#8220;$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC&#8221;, &#8220;$PROGRAM&#8221;, &#8220;$MSG&#8221;, &#8220;$SEQNUM&#8221;)<br />
indexes(&#8220;host&#8221;, &#8220;facility&#8221;, &#8220;priority&#8221;, &#8220;datetime&#8221;, &#8220;program&#8221;, &#8220;seq&#8221;));</p></blockquote>
<p>Syslog-ng connects sources, filters and destinations with <strong>log</strong> statements. The syntax is:</p>
<blockquote><p>log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); };</p></blockquote>
<p>So you have to connect my_source with my_dest:</p>
<blockquote>
<div>log { source( my_source ); destination( my_dest ); };</div>
</blockquote>
<div>If you want to test the configuration you have to restart the syslog-ng daemon and try to send a syslog event with <a title="Kiwi syslog generator" href="http://www.kiwisyslog.com/kiwi-sysloggen-download/">Kiwi Syslog Gen</a>.</div>
]]></content:encoded>
			<wfw:commentRss>http://www.iwebdev.it/blog/?feed=rss2&#038;p=163</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Collect syslog events to database (first part)</title>
		<link>http://www.iwebdev.it/blog/?p=148</link>
		<comments>http://www.iwebdev.it/blog/?p=148#comments</comments>
		<pubDate>Wed, 09 Mar 2011 22:05:32 +0000</pubDate>
		<dc:creator>to</dc:creator>
				<category><![CDATA[Log event]]></category>
		<category><![CDATA[Unix]]></category>
		<category><![CDATA[CEF syntax]]></category>
		<category><![CDATA[Common Event Format]]></category>
		<category><![CDATA[database]]></category>
		<category><![CDATA[event log]]></category>
		<category><![CDATA[syslog-ng]]></category>

		<guid isPermaLink="false">http://www.iwebdev.it/blog/?p=148</guid>
		<description><![CDATA[Syslog-ng is an open source implementation of the Syslog protocol for Unix and Unix-like systems. It extends the original syslogd model with content-based filtering, rich filtering capabilities, flexible configuration options and adds important features to syslog, like using TCP for transport. In syslog-ng starting from version 3.0 there is a great option of forward logs directly to database (Postgres, or for [...]]]></description>
			<content:encoded><![CDATA[<p><strong>Syslog-ng</strong> is an open source implementation of the Syslog protocol for Unix and Unix-like systems. It extends the original syslogd model with content-based filtering, rich filtering capabilities, flexible configuration options and adds important features to syslog, like using TCP for transport. In syslog-ng starting from version 3.0 there is a great option of <strong>forward logs directly to database</strong> (Postgres, or for that matter to MySQL, Firebird or sqlite database). In comparison with the old way of doing that, namely using a pipe and executing either a wrapper script or mysql client directly, the new way saves a great deal of resources as syslog-ng does not need to start a process every time there is a log message to log. So if you want this features you have to install syslog-ng of version 3.0 or greater with <strong>use flag sql enabled</strong>. In order to install syslog-ng you have to download the right version from the <a title="BalaBit IT Security Syslog-ng" href="http://www.balabit.com/downloads/files/syslog-ng/sources" target="_blank">official site</a>. For our purpose we download the syslog-ng 3.2.2 version (3.2.2/setups/linux-glibc2.3.6-i386).</p>
<blockquote><p>wget http://www.balabit.com/downloads/files?path=/syslog-ng/sources/3.2.2/setups/linux-glibc2.3.6-i386/syslog-ng-3.2.2-linux-glibc2.3.6-i386.run</p></blockquote>
<p>Once you downloaded the file you have to grant execute permission to syslog-ng-3.2.2-linux-glibc2.3.6-i386.run.</p>
<blockquote><p>chmod +x syslog-ng-3.2.2-linux-glibc2.3.6-i386.run</p></blockquote>
<p>Now you are ready to install the syslog-ng.</p>
<blockquote><p>./syslog-ng-3.2.2-linux-glibc2.3.6-i386.run</p></blockquote>
<p>The first screen shows the path where the syslog-ng will be installed; you have to presso &#8220;continue&#8221;.</p>
<p><a href="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-2.png"><img class="aligncenter size-medium wp-image-151" title="Syslog-ng installation step 1" src="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-2-300x180.png" alt="" width="300" height="180" /></a>The second screen resumes the parameters about your system; press &#8220;yes&#8221; if the information are corrects.</p>
<p><a href="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-3.png"><img class="aligncenter size-medium wp-image-152" title="Syslog-ng installation step 2" src="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-3-300x180.png" alt="" width="300" height="180" /></a>The third screen suggest user to check if the &#8220;/opt/syslog-ng/bin&#8221; and &#8220;/opt/syslog-ng/sbin&#8221; directory are in the <strong>search PATH</strong>. In order to do so, please add the following line into the shell profile:</p>
<blockquote><p>PATH=/opt/syslog-ng/bin:$PATH</p></blockquote>
<p><a href="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-4.png"><img class="aligncenter size-medium wp-image-153" title="Syslog-ng installation step 3" src="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-4-300x180.png" alt="" width="300" height="180" /></a>The fourth step checks if there is old version of syslog-ng installed. If the installer has detected a configuration file from a previous syslog-ng installation, the user can use this old configuration file. We choose &#8220;no&#8221;.</p>
<p><a href="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-5.png"><img class="aligncenter size-medium wp-image-154" title="Syslog-ng installation step 4" src="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-5-300x180.png" alt="" width="300" height="180" /></a>The installer generates a simple configuration file and asks if user wants to receive log messages from the network. We choose &#8220;yes&#8221;.</p>
<p><img class="aligncenter size-medium wp-image-155" title="Syslog-ng installation step 5" src="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-6-300x180.png" alt="" width="300" height="180" />The last step asks user if he wants forward the log messages to a remote server; we choose &#8220;skip&#8221;.</p>
<p><a href="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-7.png"><img class="aligncenter size-medium wp-image-156" title="Syslog-ng installation step 6" src="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-7-300x180.png" alt="" width="300" height="180" /></a></p>
<p>Congratulation, we installed syslog-ng 3.2.2.</p>
<p><a href="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-8.png"><img class="aligncenter size-medium wp-image-157" title="Syslog-ng installation final step" src="http://www.iwebdev.it/blog/wp-content/uploads/2011/03/screen-capture-8-300x180.png" alt="" width="300" height="180" /></a></p>
]]></content:encoded>
			<wfw:commentRss>http://www.iwebdev.it/blog/?feed=rss2&#038;p=148</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>
