]> git.itanic.dy.fi Git - rrdd/log
rrdd
11 years agojobcontrol: Fork bomb prevental
Timo Kokkonen [Sun, 28 Oct 2012 16:12:08 +0000 (18:12 +0200)]
jobcontrol: Fork bomb prevental

If something happens that prevents existing jobs from terminating, it
is generally bad practice to keep on forking more processes
indefinitely. To alleviate the problem, implement a somewhat trivial
process limitation feature. This will define a limit of maximum number
for the processes pending execution. If the number exceeds the global
limit, new processes are to be terminate itself immediately.

This however prevents only "limited forks" for creating too many
processes for waiting execution. Normal forks still have no
limit. This should still make the system to stay alive longer during
situations where something prevents (temporarily) executing processes
to finish in time.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agorrd images: Make the timestamp string configurable
Timo Kokkonen [Sun, 28 Oct 2012 10:45:01 +0000 (12:45 +0200)]
rrd images: Make the timestamp string configurable

Make it possible to replace the "Last update ..." string with
something that fits better for the needs of the end user. The string
is passed through strftime so one can use arbitrary timestamp formats.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoconfig: Fix copy paste errors in error messages
Timo Kokkonen [Sun, 28 Oct 2012 10:41:48 +0000 (12:41 +0200)]
config: Fix copy paste errors in error messages

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoprocess: Implement event handling infrastructure
Timo Kokkonen [Thu, 11 Oct 2012 18:30:20 +0000 (21:30 +0300)]
process: Implement event handling infrastructure

Implement proper event registration infrastucture for handling the
events coming in through the file descriptors. This makes it possible
to expand the polling code to arbitrary number of file descriptors.

All current users of the epoll have been converted to the new
infra. Some functions were moved higher in the file in order to make
it possible to reference them in the init_jobcontrol() function.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoprocess: request_fork: Ensure function parameter is always signed
Timo Kokkonen [Mon, 1 Oct 2012 14:45:52 +0000 (17:45 +0300)]
process: request_fork: Ensure function parameter is always signed

Some architectures, such as armv6, treat char as an unsigned integer
instead of signed. Basically it makes no sense to use char type with
the variable. When it is changed to an int there is no confusion about
the type and it will work better with all architectures.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agorrdtool_update_data: Print out the input data
Timo Kokkonen [Sat, 29 Sep 2012 17:09:56 +0000 (20:09 +0300)]
rrdtool_update_data: Print out the input data

It appears that it is quite often useful to see what is the actual
data that is being put into the rrd database. Therefore, print out the
data always.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoRemove extra space from debug prints
Timo Kokkonen [Tue, 17 Jul 2012 11:32:36 +0000 (14:32 +0300)]
Remove extra space from debug prints

Concatenating the two strings caused on extra space to appear on all
debug prints. Remove the extra space to make prints more pretty.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoMerge scheduler.c into rrdtool.c
Timo Kokkonen [Tue, 17 Jul 2012 11:26:22 +0000 (14:26 +0300)]
Merge scheduler.c into rrdtool.c

The schedule calculations were all very much dependent on the rrdtool
code and not something independent. It doesn't make sense to keep it
separate from the rrdtool code.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoonewire_parser: Add support for temperature offsets
Timo Kokkonen [Mon, 2 Jul 2012 18:55:24 +0000 (21:55 +0300)]
onewire_parser: Add support for temperature offsets

Some temperature sensors may have a significant systematic error
offset on their readings. This patch makes it possible to define an
offset value that is added to the raw value before the result is
stored in the rrd database.

From now on it is assumed that each onewire path never contains white
space. If there is a white space within the server path, it is
expected that each string that is separated with white space contains
an option that is used to control the readout of the values returned
or available from the server.

At the moment only one option is supported; temperature offset. The
option string must contain "offset=" following a decimal number. The
offset value is added to the number read from the server.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoscheduler: Give the name of the next db to be updated
Timo Kokkonen [Thu, 28 Jun 2012 07:55:41 +0000 (10:55 +0300)]
scheduler: Give the name of the next db to be updated

This is somewhat useful in debugging as the user will be able to know
which database will be updated next, so he can anticipate what kind of
output there is to be expected soon.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agorrdtool: Ensure data read by the parser is always NULL terminated
Timo Kokkonen [Wed, 27 Jun 2012 19:06:07 +0000 (22:06 +0300)]
rrdtool: Ensure data read by the parser is always NULL terminated

To seal out any possible denial of service attacs, do not expect the
parsers to return data that is NULL terminated.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoscript_parser: Fix fread to return number of bytes read
Timo Kokkonen [Wed, 27 Jun 2012 19:00:50 +0000 (22:00 +0300)]
script_parser: Fix fread to return number of bytes read

From fread man page:

The function fread() reads nmemb elements of data, each size bytes
long, from the stream pointed to by stream, storing them at the
location given by ptr.

If size is something else than 1, the return value is not number of
bytes read. The intention here is to have it return number of bytes so
that the debug print can print it correctly.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoprocess: Print process output through the debug macros
Timo Kokkonen [Wed, 27 Jun 2012 15:35:50 +0000 (18:35 +0300)]
process: Print process output through the debug macros

As we are appending our own prefixes to all process prints, we might
aswell use the normal debug macros to unify the prints with the rest
of the debug output.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoparser: Remove old debug print
Timo Kokkonen [Wed, 27 Jun 2012 15:29:14 +0000 (18:29 +0300)]
parser: Remove old debug print

All debug prints should use the pr_* variants instead of printf
directly. This one print in question is not very useful, so it can be
removed altogether.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoprocess: Fix bug with child processes stuck forever when parent segfaults
Timo Kokkonen [Wed, 27 Jun 2012 15:18:19 +0000 (18:18 +0300)]
process: Fix bug with child processes stuck forever when parent segfaults

After a fork() the child needs to close the unused end of the process
control pipes. Failing to do so will prevent EOF and EPIPE signals
from being delivered correctly back to the child when the parent
closes its end of the pipe.

Furthermore, the code's assumption about the behaviour of the pipe
with one writer and multiple readers was wrong. When there are
multiple readers, only one will be woken up and the rest will be left
blocking the read in case there was nothing left to read. If read
returns zero, that means EOF, eg. the job control parent has closed
its end of the pipe.

These fixes applied there should be no more problem with a lot of
child processes stuck in waiting the parent to give them a permission
to go. They will either fail with SIGPIPE when they request the
permission to run or they will read zero bytes from the pipe and
continue executing instantly.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agomain: Print out the time when next even it expected
Timo Kokkonen [Tue, 26 Jun 2012 19:17:30 +0000 (22:17 +0300)]
main: Print out the time when next even it expected

This is useful during debugging and we want to know when the next
event begins.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agodebug: Reintroduce message colouring
Timo Kokkonen [Tue, 26 Jun 2012 13:40:58 +0000 (16:40 +0300)]
debug: Reintroduce message colouring

When the debug macros were refactored into a separate function, all
messages were printed with green color. Change it to use the desired
color instead.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agodebug.c: Remove indentation from the prints
Timo Kokkonen [Tue, 26 Jun 2012 13:33:14 +0000 (16:33 +0300)]
debug.c: Remove indentation from the prints

The child-depth indentation is not really that useful. In fact, it
makes actually harder to read similar debug print lines with different
indentation. The numerical level of the child generation gives
sufficient information.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agodebug: Rework debugging
Timo Kokkonen [Sun, 24 Jun 2012 19:59:23 +0000 (22:59 +0300)]
debug: Rework debugging

Instead of having a huge debug macro that gets expanded every time a
debug print is needed, separate the debugging facilities into
debu.c. This will reduce excess code duplication a lot.

A concept of debugging level is also introduced. This makes it
possible to add debug prints with different priority level and have
some of the prints to be silenced when user is not interested about
them.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoAdd missing headers
Timo Kokkonen [Sun, 24 Jun 2012 19:57:25 +0000 (22:57 +0300)]
Add missing headers

These files were depending on headers from debug.h. The debug.h is
going to be cleaned up and these files no longer compile without these
headers.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoAdd onewire parser
Timo Kokkonen [Sun, 24 Jun 2012 08:47:25 +0000 (11:47 +0300)]
Add onewire parser

This parser will parse data from onewire server. It will use the
network protocol to query the data, thus it is capable of fetching the
data over any network location supported by the ownet library.

The first parser_data string must containt the server address:ip. The
rest of the strings are used to query data entries from the server. If
one wishes to leave blank entries to some of the data, a "U" string is
passed as is to the rrd_data.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoAdd MIN() macro
Timo Kokkonen [Sun, 24 Jun 2012 08:54:35 +0000 (11:54 +0300)]
Add MIN() macro

As we don't have yet any header for random useful macros, add this
here so that parsers can utilize it.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agorrdtool.c: Improve data sanitization
Timo Kokkonen [Sun, 24 Jun 2012 08:29:27 +0000 (11:29 +0300)]
rrdtool.c: Improve data sanitization

The code is doing rather complex string manipulation. Without
sufficient commentation it is hard to understand what it really
does. Better documentation is now included.

Non-numerical data is also now marked as undefined, if there are no
numbers at all in the input data. Completely empty entries are also
marked as undefined.

Finally, santitized string output is NULL terminated even if also the
last entry was undefined.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoChange parser parameter type
Timo Kokkonen [Sat, 23 Jun 2012 12:17:25 +0000 (15:17 +0300)]
Change parser parameter type

As the parser parameter is coming from the config file now and it is
defined as a list of strings there, the type of the parameter on the
parser function type should match it.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoconfig.c: Fix macro variable usage
Timo Kokkonen [Sat, 23 Jun 2012 10:46:05 +0000 (13:46 +0300)]
config.c: Fix macro variable usage

The variable name was missing an underscore, thus it was using the
variable outside of its parameter list.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoEnable database reading and writing, if requested from command line
Timo Kokkonen [Wed, 20 Jun 2012 19:59:27 +0000 (22:59 +0300)]
Enable database reading and writing, if requested from command line

If no command line arguments are given, continue using the old built
in database config. But if appropriate config is given as a command
line argument, read the entries from the file. If the user does not
have a database config file to get started with, make it possible for
the default database to be created as a starting point.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoAdd Initial version of config file handling
Timo Kokkonen [Wed, 20 Jun 2012 19:55:42 +0000 (22:55 +0300)]
Add Initial version of config file handling

Support for reading and writing database entries from and to config
files. Can handle most of the normal cases.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoRename the built in database to default_rrds
Timo Kokkonen [Wed, 20 Jun 2012 19:50:34 +0000 (22:50 +0300)]
Rename the built in database to default_rrds

In future this is not going to be the database in use, but the default
one that contains the data that can be used to dump the default
database list. Thus the change in the name.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agorrdtool.c: Improve error handling during database creation
Timo Kokkonen [Wed, 20 Jun 2012 19:22:23 +0000 (22:22 +0300)]
rrdtool.c: Improve error handling during database creation

The case of filename pointer being NULL was not handled. The logic for
testing whether the database file existed at all should not return
true in case the file name is zero. As this function can't return
failure code, the error of NULL filename pointer is handled later when
the database is being created.

As many missing databases are created as possible. In case of errors
we will continue hoping that we will be able to update at least
something.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agonetstats_parser: Fix crash when network device names are undefined
Timo Kokkonen [Wed, 20 Jun 2012 19:19:25 +0000 (22:19 +0300)]
netstats_parser: Fix crash when network device names are undefined

There is absolutely nothing this parser can do without the network
names. If the pointer happens to be zero, all we can do is to bail out
with a error print. Not handling this situation will lead to a crash.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agorrdtool.h: Constify string constants
Timo Kokkonen [Mon, 4 Jun 2012 15:42:58 +0000 (18:42 +0300)]
rrdtool.h: Constify string constants

String constants are being stored to these fields, so they should be
constants. Relacant casts and other changes are made within all users
of these structures.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agomain: Add NULL terminator to option array
Timo Kokkonen [Sat, 19 May 2012 18:06:49 +0000 (21:06 +0300)]
main: Add NULL terminator to option array

getopt_long() requires the struct option array to be terminated with
an element that is filled with zeroes.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoprocess: Replace racy SIGCHLD handler with signalfd
Timo Kokkonen [Tue, 15 May 2012 19:13:25 +0000 (22:13 +0300)]
process: Replace racy SIGCHLD handler with signalfd

Having an asynchronous signal handler for SIGCHLD handling is quite
racy. We would like to be able to print debug messages when reaping
children, but we cannot use printf or other signal unsafe function
calls.

Signalfd is the solution that works. As we already have event loop
with epoll, we can extend it to read the signals via the
descriptor. And there is also one place less for causing potential
race conditions.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoprocess: Improve harvest_zombies
Timo Kokkonen [Tue, 15 May 2012 19:07:18 +0000 (22:07 +0300)]
process: Improve harvest_zombies

Use wait4 instead of waitpid. This makes it possible to get resource
usage information from the child that exited. The user and system time
is printed out for curious developer to inspect.

If a child happens to be stopped or continued, we are now ignoring
those state changes. This prevents zombie processes to be left in case
something special happens to the child.

Further improvement is that it is possible to tell whether the child
was killed with some signal or if it exited with a code.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoprocess.c: Factor out job request handling
Timo Kokkonen [Sun, 13 May 2012 15:06:45 +0000 (18:06 +0300)]
process.c: Factor out job request handling

Separate the epoll_wait handling and the actual file descriptor
handling. This is preparation for signald_fd handling.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agomain: Exit on init_jobcontrol failure
Timo Kokkonen [Sat, 12 May 2012 19:41:43 +0000 (22:41 +0300)]
main: Exit on init_jobcontrol failure

If we fail to initialize critical structures, there is no point in
proceeding any further.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoprocess.c: Move epoll initialization into init_jobcontrol()
Timo Kokkonen [Sat, 12 May 2012 19:17:40 +0000 (22:17 +0300)]
process.c: Move epoll initialization into init_jobcontrol()

This is where the rest of the job control structures are initialized,
this should be initialized there too.

The missing epoll_ctl error handling is also now in.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
11 years agoRename init_max_jobs() to init_jobcontrol()
Timo Kokkonen [Sat, 12 May 2012 19:14:00 +0000 (22:14 +0300)]
Rename init_max_jobs() to init_jobcontrol()

This describes better what the function does; it initializes the
entire jobcontrol into functional state. That is more than setting the
number of maximum jobs.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agomain: Implement command line parsing
Timo Kokkonen [Wed, 25 Apr 2012 15:22:34 +0000 (18:22 +0300)]
main: Implement command line parsing

The first command line option we support is the number of jobs to run
in parallel.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agorrdtool: Set up the correct executable name when running processes
Timo Kokkonen [Sun, 22 Apr 2012 18:46:42 +0000 (21:46 +0300)]
rrdtool: Set up the correct executable name when running processes

The first argument, the name of the executable, should be set properly
prior running execv(). This ensures the process knows properly how it
was called. There might be a difference in case the process implements
multiple personalities based on the name it was called.

This does not appear to be the case with rrdtool as it was working
even if the executable name was left blank.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agoprocess: sigchild_handler: Only call waitpid()
Timo Kokkonen [Sun, 15 Apr 2012 19:03:21 +0000 (22:03 +0300)]
process: sigchild_handler: Only call waitpid()

Calling any *printf* functions from a signal handler is unsafe. If the
interrupted process happened to be in the middle of a printf call
during the arrival of the signal, the signal handler can freeze
silently.

To avoid this, the sigchild handler will no longer call
harvest_zombies() to clear out the zombie processes. This hides the
exit status of the process, but that is not that significant
information.

An alternative way would be to simply do nothing in the signal handler
except set some flag indicating the death of a child. The signal would
cause the epoll_wait to fail with EINTR error that can be used to
trigger waitpid.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agodebug: Flush debug prints out immedeatly
Timo Kokkonen [Sun, 15 Apr 2012 18:21:37 +0000 (21:21 +0300)]
debug: Flush debug prints out immedeatly

This should ensure log output is free of possible corruptions due to
stdio buffering.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agorrdtool_draw_image: Draw images in parallel
Timo Kokkonen [Sun, 15 Apr 2012 18:20:35 +0000 (21:20 +0300)]
rrdtool_draw_image: Draw images in parallel

Use the do_fork_limited() to draw images in parallel while avoiding to
draw too many images at the same time.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agomain.c: Enable job control
Timo Kokkonen [Sun, 15 Apr 2012 18:18:39 +0000 (21:18 +0300)]
main.c: Enable job control

Start using the new job control feature. No need to sleep any more
fixed sleeps. The job scheduler will use epoll_wait timeout for
sleeping instead.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agoprocess.c: Implement support for limiting number of active processes
Timo Kokkonen [Sun, 15 Apr 2012 18:08:26 +0000 (21:08 +0300)]
process.c: Implement support for limiting number of active processes

This will make it possible to have only limited number of active job
processes runnint at given time. These can be requested by calling
do_fork_limited(), which works otherwise similarly to do_fork() but
the child process will not start running until the main parent has
given the child permission to run.

The job controlling is implemented via pipes between the parent and
the children. The child which wish to limit the number of processes
will send its pid to the parent. The master parent will keep count of
all the processes running. If the number of active processes grows too
high, no new jobs are granted until fewer processes are running. Once
the parent decides that a new job can become active, it will write one
byte to a pipe. The child which reads the byte out is the one who has
the right to execute. Other children reading the same pipe are left
waiting for their turn.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agoprocess.c: Remove the rest of the printf() calls
Timo Kokkonen [Sun, 15 Apr 2012 17:31:07 +0000 (20:31 +0300)]
process.c: Remove the rest of the printf() calls

No reason to not unify the prints whenever possible.

However, the process stdout and stderr prints are special. They do not
come from rrdd, but from the process that was execv'd by rrdd. That
why they should begin with the name of the binary. Those prints should
be printed to stderr too to unify the debug output.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agoprocess: Remove get_sibling_count()
Timo Kokkonen [Sun, 15 Apr 2012 17:26:02 +0000 (20:26 +0300)]
process: Remove get_sibling_count()

This function does not really provide any useful information. It was
used only in the debug prints, where it was printed after the pid
number. However, it is much more useful to print the parent count
instead to see how deep we are in the fork chain.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agorrdtool: Improve debug prints
Timo Kokkonen [Sat, 14 Apr 2012 17:39:19 +0000 (20:39 +0300)]
rrdtool: Improve debug prints

Use the debug macro for printing. Also remove the unimportant message
indicating the job is done. It is obvious that all is done when other
prints start coming out.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agorrdtool: Rename rrdtool_check_databases() to rrdtool_create_missing_databases()
Timo Kokkonen [Sat, 14 Apr 2012 17:38:05 +0000 (20:38 +0300)]
rrdtool: Rename rrdtool_check_databases() to rrdtool_create_missing_databases()

This name is more descriptive with respect to what the function
actually does.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agoscheduler: Rename check_update_need() to get_outdated_db()
Timo Kokkonen [Sat, 14 Apr 2012 17:32:34 +0000 (20:32 +0300)]
scheduler: Rename check_update_need() to get_outdated_db()

The new name is more descriptive with respect what the function
actually does.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agoRemove redundant strerror usage
Timo Kokkonen [Sat, 14 Apr 2012 12:06:52 +0000 (15:06 +0300)]
Remove redundant strerror usage

It is preferred to use '%m' with printf instead.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agoprocess: Improve error handling
Timo Kokkonen [Sat, 14 Apr 2012 12:01:19 +0000 (15:01 +0300)]
process: Improve error handling

Check the error message from select(). If error, bail out.

Also close all file descriptors in the end. If select happends to fail
and we go to waitpid() and the process is writing content, it might
eventually block because nobody is reading the file descriptor. By
closing the file we ensure the process will not hang and there will be
no deadlock.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agorrdtool: Reduce excess parallelism
Timo Kokkonen [Thu, 12 Apr 2012 18:52:09 +0000 (21:52 +0300)]
rrdtool: Reduce excess parallelism

Right now all commands are run "as parallel as possible", up to the
point images are being drawn while the databases are still
updated. This leads to problems where the image drawing finishes
before databases are read, thus the image contains always the
information from the last update instead of current.

This patch modifies the behavior so that when the database is being
updated, the drawing does not beging before the update has
succeeded.

Furthermore, image drawings are not done fully parallel. If they were,
the temporary file would be always renamed before there was any chance
that the actual image rendering was finished. Instead, rename is
synchronized with the image drawing.

As a consequence total parallelism is greatly reduced. This obviously
slows down update and image drawing process greatly on machines that
have many CPUs, that would otherwise benefit greatly from being able
to all actions in parallel. The maximum number of parallel actions is
practically limited to the number of databases that are being operated
with. The correctness of the operations is however more important than
maximising parallelism.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agodebug.h: Add timestamps to debug prints
Timo Kokkonen [Thu, 12 Apr 2012 18:47:24 +0000 (21:47 +0300)]
debug.h: Add timestamps to debug prints

This is useful when someone needs to run rrdd for long time and debug
the behavior from logs. Especially timing issues become easier to
understand once the timestamps are visible.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agorrdtool_draw_image: Avoid corrupting image files
Timo Kokkonen [Wed, 11 Apr 2012 17:46:29 +0000 (20:46 +0300)]
rrdtool_draw_image: Avoid corrupting image files

As rrdtool is creating the image file it does appear to modify always
the contents of the existing file. This leads to a potential race
condition where someone is reading the current image contents and then
rrdtool changes the contents while the reader is still in the process
of the reading the old contents. If that happens, the image file
appears to be corrupted due to the fact that it contains a mixture of
the old and new image data.

Fix the issue by creating a temporary file where the image is created
and then replacing the target file with the temp file only after
rrdtool has finished drawing all of the image. This ensures reader
will be accessing either the old or new image contents, newer
something that is a mixture of those two.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agoAdd "other" memory counter
Timo Kokkonen [Wed, 22 Feb 2012 20:09:33 +0000 (22:09 +0200)]
Add "other" memory counter

The current memory counters taken from proc/meminfo don't add up
taking the entire usable physical memory. There is always a little bit
of memory left out from the equation that doesn't sum up. In some
situations that can be actualyl quite significant. Now this "Other"
memory has its own counter and the total amount of height of the
memory graph stays constant over time.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agoscript_parser: Print number of read bytes in the info message
Timo Kokkonen [Wed, 22 Feb 2012 20:07:20 +0000 (22:07 +0200)]
script_parser: Print number of read bytes in the info message

This is not strictly necessary, but might be useful. It also hides the
compiler warning about setting a variable but not reading it ever that
has become visible since gcc-4.6.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agomem_parser: Replace zero pointer with NULL
Timo Kokkonen [Wed, 22 Feb 2012 20:04:13 +0000 (22:04 +0200)]
mem_parser: Replace zero pointer with NULL

This is more grammatically correc.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agostring.c: Use existing string helpers
Timo Kokkonen [Wed, 22 Feb 2012 20:02:02 +0000 (22:02 +0200)]
string.c: Use existing string helpers

Since we have those fancy helpers, why not use them? Also, add one
extra for skipping float nubers for extra handiness.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agostring.h: Use defintions from ctype.h when applicable
Timo Kokkonen [Wed, 22 Feb 2012 19:59:49 +0000 (21:59 +0200)]
string.h: Use defintions from ctype.h when applicable

This improves code readability and maintainability without costing
anything.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agodatabase.h: Clean up default path names
Timo Kokkonen [Wed, 22 Feb 2012 16:05:43 +0000 (18:05 +0200)]
database.h: Clean up default path names

Replace the default SYSINFO_PATH with something more generic. Also,
apply the path to whereever it applies.

Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
12 years agodatabase.h: Add time variable to DEFINE_IMAGE macro
Timo Kokkonen [Sat, 3 Sep 2011 07:49:34 +0000 (10:49 +0300)]
database.h: Add time variable to DEFINE_IMAGE macro

It will make it possible to conviniently define how many
days/weeks/months/years to draw in the image. It is no longer needed
to have the unit fixed to 1.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
12 years agosanitize_rrd_update_data: Ensure minus sign is not forgotten
Timo Kokkonen [Sat, 25 Jun 2011 11:25:20 +0000 (14:25 +0300)]
sanitize_rrd_update_data: Ensure minus sign is not forgotten

Fix a simple bug which caused the minus sign to be removed from the
sanitized output.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoparser: Use long long when parsing net stats
Timo Kokkonen [Wed, 6 Apr 2011 14:57:49 +0000 (17:57 +0300)]
parser: Use long long when parsing net stats

It might be sufficient to use unsigned long in 32 bit environment, but
in order to avoid any issues with overflows, long long is safer.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agodatabase.h: Multiple fixes
Timo Kokkonen [Sun, 3 Apr 2011 09:54:36 +0000 (12:54 +0300)]
database.h: Multiple fixes

-Network rate multiplier changes dynamically (k, M, G)

-Memory statistics base is 1024 instead of 1000

-Network autoscale minimum is 0

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoRename database.h.example to database.h
Timo Kokkonen [Sun, 3 Apr 2011 09:49:54 +0000 (12:49 +0300)]
Rename database.h.example to database.h

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agorrdtool_update_data: Sanitize input data before passing to rrdtool
Timo Kokkonen [Sun, 3 Apr 2011 09:21:16 +0000 (12:21 +0300)]
rrdtool_update_data: Sanitize input data before passing to rrdtool

If there is any non-numberic data in the input data, replace them with
'U' so that rrdtool doesn't refuse the entire input data set.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agostring: Add functions to skip numbers and non-numbers
Timo Kokkonen [Sun, 3 Apr 2011 09:20:10 +0000 (12:20 +0300)]
string: Add functions to skip numbers and non-numbers

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agodatabase.h.example: Add network statistics
Timo Kokkonen [Sun, 27 Mar 2011 20:19:34 +0000 (23:19 +0300)]
database.h.example: Add network statistics

Read statistics for eth0 and eth0

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agodatabase.h.example: Add macro for generating rrd source and archives
Timo Kokkonen [Sun, 27 Mar 2011 20:16:24 +0000 (23:16 +0300)]
database.h.example: Add macro for generating rrd source and archives

This reduces significantly the numer of lines in the database.h file.
It also improves readability significantly due to reduced line number
count.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoparser: Add network parser
Timo Kokkonen [Sun, 27 Mar 2011 19:52:52 +0000 (22:52 +0300)]
parser: Add network parser

This can be used for parsing network statistics from /proc/net/dev
interface. The desired interface is defined by giving string pointers
to interface names.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agodec_to_longlong: Return values should be long long
Timo Kokkonen [Sun, 27 Mar 2011 16:04:27 +0000 (19:04 +0300)]
dec_to_longlong: Return values should be long long

Otherwise the return value might warp aroung.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agodatabase.h.example: Add macro to fill rrd_image structures
Timo Kokkonen [Sun, 13 Mar 2011 09:27:00 +0000 (11:27 +0200)]
database.h.example: Add macro to fill rrd_image structures

This reduces massively the number of lines and makes it much easier to
add new images.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agorrdtool: Print out the image that is being drawn
Timo Kokkonen [Sun, 13 Mar 2011 09:20:53 +0000 (11:20 +0200)]
rrdtool: Print out the image that is being drawn

This makes debugging easier in case some of the images get drawn
correctly and some don't.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agodatabase.h.example: Update example data
Timo Kokkonen [Fri, 11 Mar 2011 21:20:43 +0000 (23:20 +0200)]
database.h.example: Update example data

Some typo fixes and improvements

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoparser: Add support for a script parser
Timo Kokkonen [Fri, 11 Mar 2011 21:16:51 +0000 (23:16 +0200)]
parser: Add support for a script parser

This parser will simply run some command and assume the command writes
the rrd tool data as is to stdout.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoprocess.c: Ensure stderr gets read after process has died
Timo Kokkonen [Fri, 11 Mar 2011 20:07:56 +0000 (22:07 +0200)]
process.c: Ensure stderr gets read after process has died

For some reason select() doesn't indicate that stderr might contain
bytes that can be read. Workaround for the issue is to read stderr
just in case after we receive EOF from stdin.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoparser: Define maximum length for the rrd data
Timo Kokkonen [Fri, 11 Mar 2011 13:50:43 +0000 (15:50 +0200)]
parser: Define maximum length for the rrd data

This makes makes buffer overruns harder.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoparser: Add extra pointer to parse function prototype
Timo Kokkonen [Fri, 11 Mar 2011 13:20:13 +0000 (15:20 +0200)]
parser: Add extra pointer to parse function prototype

This pointer makes it possible to feed some data to the parser, such
as command line parameters.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agodatabase.h.example: Add information to allow database creation
Timo Kokkonen [Mon, 7 Mar 2011 16:00:28 +0000 (18:00 +0200)]
database.h.example: Add information to allow database creation

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoMakefile: Typo fix
Timo Kokkonen [Mon, 7 Mar 2011 15:52:49 +0000 (17:52 +0200)]
Makefile: Typo fix

Make quiet linkin work

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoAdd support for creating rrd databases in case the file is missing
Timo Kokkonen [Sun, 6 Mar 2011 17:05:24 +0000 (19:05 +0200)]
Add support for creating rrd databases in case the file is missing

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agorrdtool: Improve argument list modification macro
Timo Kokkonen [Mon, 7 Mar 2011 08:24:08 +0000 (10:24 +0200)]
rrdtool: Improve argument list modification macro

The name of the macro was not very descriptive, "add_arg" describes
the functionality much better.

Instead of modifying some magic function variables, give the variables
as parameters to the macro. This makes it much more clear how the
thing actually works.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agorrdtool: Replace repeating strings with a definition
Timo Kokkonen [Mon, 7 Mar 2011 08:20:15 +0000 (10:20 +0200)]
rrdtool: Replace repeating strings with a definition

The command path should be in a defintion, in case one wants to change
it.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoMove rrd_database structure under rrdtool.h
Timo Kokkonen [Sun, 6 Mar 2011 16:48:13 +0000 (18:48 +0200)]
Move rrd_database structure under rrdtool.h

Keep all rrdtool related structure definitions under this one header
file.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoCollect all rrdtool handling in one file
Timo Kokkonen [Sun, 6 Mar 2011 16:31:00 +0000 (18:31 +0200)]
Collect all rrdtool handling in one file

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agodatabase.h.example: Avoid pathname repeatitions
Timo Kokkonen [Sun, 6 Mar 2011 13:13:43 +0000 (15:13 +0200)]
database.h.example: Avoid pathname repeatitions

This way it is much easier to modify the default paths.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoRename testadata.h to database.h
Timo Kokkonen [Sun, 6 Mar 2011 12:54:54 +0000 (14:54 +0200)]
Rename testadata.h to database.h

This is probably a more descriptive name.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
13 years agoMakefile: Fixes
Timo Kokkonen [Sun, 6 Mar 2011 10:19:43 +0000 (12:19 +0200)]
Makefile: Fixes

Proper dependency checking. Improved pretty printing.

Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
14 years agoOrthogonalize CPU graph colors
Timo Kokkonen [Thu, 8 Oct 2009 19:41:35 +0000 (22:41 +0300)]
Orthogonalize CPU graph colors

Otherwise IO and USER load might get mixed up

14 years agoChange default memory graph colors
Timo Kokkonen [Thu, 8 Oct 2009 17:11:20 +0000 (20:11 +0300)]
Change default memory graph colors

Swap is better to be in red, as red is "bad". Thus, buffers and swap
colors are now reversed.

14 years agoUse long long instead of ints when parsing cpu tics
Timo Kokkonen [Thu, 1 Oct 2009 11:15:10 +0000 (14:15 +0300)]
Use long long instead of ints when parsing cpu tics

This fixes a problem with overwrapping counters on 32bit architectures.

14 years agoUpdate testdata.h.example to support swap
Timo Kokkonen [Sun, 20 Sep 2009 16:53:45 +0000 (19:53 +0300)]
Update testdata.h.example to support swap

14 years agoAdd support for reading swap usage from meminfo
Timo Kokkonen [Thu, 17 Sep 2009 19:27:02 +0000 (22:27 +0300)]
Add support for reading swap usage from meminfo

15 years agoPrint which command is to be executed
Timo Kokkonen [Wed, 21 May 2008 17:39:44 +0000 (20:39 +0300)]
Print which command is to be executed

15 years agoChange debug message output
Timo Kokkonen [Wed, 21 May 2008 17:18:46 +0000 (20:18 +0300)]
Change debug message output

15 years agoRename get_process_count() to get_sibling_count()
Timo Kokkonen [Wed, 21 May 2008 17:15:41 +0000 (20:15 +0300)]
Rename get_process_count() to get_sibling_count()

15 years agoUse pointers properly
Timo Kokkonen [Wed, 21 May 2008 17:12:43 +0000 (20:12 +0300)]
Use pointers properly

15 years agoAdd primitive error detection to digitemp parser
Timo Kokkonen [Wed, 21 May 2008 17:11:07 +0000 (20:11 +0300)]
Add primitive error detection to digitemp parser

15 years agoFix debug message indenting bug
Timo Kokkonen [Wed, 21 May 2008 16:51:14 +0000 (19:51 +0300)]
Fix debug message indenting bug