Timo Kokkonen [Wed, 20 Jun 2012 19:59:27 +0000 (22:59 +0300)]
Enable database reading and writing, if requested from command line
If no command line arguments are given, continue using the old built
in database config. But if appropriate config is given as a command
line argument, read the entries from the file. If the user does not
have a database config file to get started with, make it possible for
the default database to be created as a starting point.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 20 Jun 2012 19:50:34 +0000 (22:50 +0300)]
Rename the built in database to default_rrds
In future this is not going to be the database in use, but the default
one that contains the data that can be used to dump the default
database list. Thus the change in the name.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 20 Jun 2012 19:22:23 +0000 (22:22 +0300)]
rrdtool.c: Improve error handling during database creation
The case of filename pointer being NULL was not handled. The logic for
testing whether the database file existed at all should not return
true in case the file name is zero. As this function can't return
failure code, the error of NULL filename pointer is handled later when
the database is being created.
As many missing databases are created as possible. In case of errors
we will continue hoping that we will be able to update at least
something.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 20 Jun 2012 19:19:25 +0000 (22:19 +0300)]
netstats_parser: Fix crash when network device names are undefined
There is absolutely nothing this parser can do without the network
names. If the pointer happens to be zero, all we can do is to bail out
with a error print. Not handling this situation will lead to a crash.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Mon, 4 Jun 2012 15:42:58 +0000 (18:42 +0300)]
rrdtool.h: Constify string constants
String constants are being stored to these fields, so they should be
constants. Relacant casts and other changes are made within all users
of these structures.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 15 May 2012 19:13:25 +0000 (22:13 +0300)]
process: Replace racy SIGCHLD handler with signalfd
Having an asynchronous signal handler for SIGCHLD handling is quite
racy. We would like to be able to print debug messages when reaping
children, but we cannot use printf or other signal unsafe function
calls.
Signalfd is the solution that works. As we already have event loop
with epoll, we can extend it to read the signals via the
descriptor. And there is also one place less for causing potential
race conditions.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 15 May 2012 19:07:18 +0000 (22:07 +0300)]
process: Improve harvest_zombies
Use wait4 instead of waitpid. This makes it possible to get resource
usage information from the child that exited. The user and system time
is printed out for curious developer to inspect.
If a child happens to be stopped or continued, we are now ignoring
those state changes. This prevents zombie processes to be left in case
something special happens to the child.
Further improvement is that it is possible to tell whether the child
was killed with some signal or if it exited with a code.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 12 May 2012 19:14:00 +0000 (22:14 +0300)]
Rename init_max_jobs() to init_jobcontrol()
This describes better what the function does; it initializes the
entire jobcontrol into functional state. That is more than setting the
number of maximum jobs.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 22 Apr 2012 18:46:42 +0000 (21:46 +0300)]
rrdtool: Set up the correct executable name when running processes
The first argument, the name of the executable, should be set properly
prior running execv(). This ensures the process knows properly how it
was called. There might be a difference in case the process implements
multiple personalities based on the name it was called.
This does not appear to be the case with rrdtool as it was working
even if the executable name was left blank.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 15 Apr 2012 19:03:21 +0000 (22:03 +0300)]
process: sigchild_handler: Only call waitpid()
Calling any *printf* functions from a signal handler is unsafe. If the
interrupted process happened to be in the middle of a printf call
during the arrival of the signal, the signal handler can freeze
silently.
To avoid this, the sigchild handler will no longer call
harvest_zombies() to clear out the zombie processes. This hides the
exit status of the process, but that is not that significant
information.
An alternative way would be to simply do nothing in the signal handler
except set some flag indicating the death of a child. The signal would
cause the epoll_wait to fail with EINTR error that can be used to
trigger waitpid.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 15 Apr 2012 18:08:26 +0000 (21:08 +0300)]
process.c: Implement support for limiting number of active processes
This will make it possible to have only limited number of active job
processes runnint at given time. These can be requested by calling
do_fork_limited(), which works otherwise similarly to do_fork() but
the child process will not start running until the main parent has
given the child permission to run.
The job controlling is implemented via pipes between the parent and
the children. The child which wish to limit the number of processes
will send its pid to the parent. The master parent will keep count of
all the processes running. If the number of active processes grows too
high, no new jobs are granted until fewer processes are running. Once
the parent decides that a new job can become active, it will write one
byte to a pipe. The child which reads the byte out is the one who has
the right to execute. Other children reading the same pipe are left
waiting for their turn.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 15 Apr 2012 17:31:07 +0000 (20:31 +0300)]
process.c: Remove the rest of the printf() calls
No reason to not unify the prints whenever possible.
However, the process stdout and stderr prints are special. They do not
come from rrdd, but from the process that was execv'd by rrdd. That
why they should begin with the name of the binary. Those prints should
be printed to stderr too to unify the debug output.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 15 Apr 2012 17:26:02 +0000 (20:26 +0300)]
process: Remove get_sibling_count()
This function does not really provide any useful information. It was
used only in the debug prints, where it was printed after the pid
number. However, it is much more useful to print the parent count
instead to see how deep we are in the fork chain.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 14 Apr 2012 17:39:19 +0000 (20:39 +0300)]
rrdtool: Improve debug prints
Use the debug macro for printing. Also remove the unimportant message
indicating the job is done. It is obvious that all is done when other
prints start coming out.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 14 Apr 2012 12:01:19 +0000 (15:01 +0300)]
process: Improve error handling
Check the error message from select(). If error, bail out.
Also close all file descriptors in the end. If select happends to fail
and we go to waitpid() and the process is writing content, it might
eventually block because nobody is reading the file descriptor. By
closing the file we ensure the process will not hang and there will be
no deadlock.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Thu, 12 Apr 2012 18:52:09 +0000 (21:52 +0300)]
rrdtool: Reduce excess parallelism
Right now all commands are run "as parallel as possible", up to the
point images are being drawn while the databases are still
updated. This leads to problems where the image drawing finishes
before databases are read, thus the image contains always the
information from the last update instead of current.
This patch modifies the behavior so that when the database is being
updated, the drawing does not beging before the update has
succeeded.
Furthermore, image drawings are not done fully parallel. If they were,
the temporary file would be always renamed before there was any chance
that the actual image rendering was finished. Instead, rename is
synchronized with the image drawing.
As a consequence total parallelism is greatly reduced. This obviously
slows down update and image drawing process greatly on machines that
have many CPUs, that would otherwise benefit greatly from being able
to all actions in parallel. The maximum number of parallel actions is
practically limited to the number of databases that are being operated
with. The correctness of the operations is however more important than
maximising parallelism.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Thu, 12 Apr 2012 18:47:24 +0000 (21:47 +0300)]
debug.h: Add timestamps to debug prints
This is useful when someone needs to run rrdd for long time and debug
the behavior from logs. Especially timing issues become easier to
understand once the timestamps are visible.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 11 Apr 2012 17:46:29 +0000 (20:46 +0300)]
rrdtool_draw_image: Avoid corrupting image files
As rrdtool is creating the image file it does appear to modify always
the contents of the existing file. This leads to a potential race
condition where someone is reading the current image contents and then
rrdtool changes the contents while the reader is still in the process
of the reading the old contents. If that happens, the image file
appears to be corrupted due to the fact that it contains a mixture of
the old and new image data.
Fix the issue by creating a temporary file where the image is created
and then replacing the target file with the temp file only after
rrdtool has finished drawing all of the image. This ensures reader
will be accessing either the old or new image contents, newer
something that is a mixture of those two.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 22 Feb 2012 20:09:33 +0000 (22:09 +0200)]
Add "other" memory counter
The current memory counters taken from proc/meminfo don't add up
taking the entire usable physical memory. There is always a little bit
of memory left out from the equation that doesn't sum up. In some
situations that can be actualyl quite significant. Now this "Other"
memory has its own counter and the total amount of height of the
memory graph stays constant over time.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 22 Feb 2012 20:07:20 +0000 (22:07 +0200)]
script_parser: Print number of read bytes in the info message
This is not strictly necessary, but might be useful. It also hides the
compiler warning about setting a variable but not reading it ever that
has become visible since gcc-4.6.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 3 Sep 2011 07:49:34 +0000 (10:49 +0300)]
database.h: Add time variable to DEFINE_IMAGE macro
It will make it possible to conviniently define how many
days/weeks/months/years to draw in the image. It is no longer needed
to have the unit fixed to 1.
Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
Timo Kokkonen [Sat, 3 Sep 2011 07:49:34 +0000 (10:49 +0300)]
database.h: Add time variable to DEFINE_IMAGE macro
It will make it possible to conviniently define how many
days/weeks/months/years to draw in the image. It is no longer needed
to have the unit fixed to 1.
Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
Timo Kokkonen [Sun, 27 Mar 2011 19:52:52 +0000 (22:52 +0300)]
parser: Add network parser
This can be used for parsing network statistics from /proc/net/dev
interface. The desired interface is defined by giving string pointers
to interface names.
Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
Timo Kokkonen [Sun, 27 Mar 2011 19:52:52 +0000 (22:52 +0300)]
parser: Add network parser
This can be used for parsing network statistics from /proc/net/dev
interface. The desired interface is defined by giving string pointers
to interface names.
Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
Timo Kokkonen [Fri, 11 Mar 2011 20:07:56 +0000 (22:07 +0200)]
process.c: Ensure stderr gets read after process has died
For some reason select() doesn't indicate that stderr might contain
bytes that can be read. Workaround for the issue is to read stderr
just in case after we receive EOF from stdin.
Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>
Timo Kokkonen [Fri, 11 Mar 2011 20:07:56 +0000 (22:07 +0200)]
process.c: Ensure stderr gets read after process has died
For some reason select() doesn't indicate that stderr might contain
bytes that can be read. Workaround for the issue is to read stderr
just in case after we receive EOF from stdin.
Signed-off-by: Timo Kokkonen <kaapeli@itanic.dy.fi>