Timo Kokkonen [Wed, 30 Sep 2020 17:44:37 +0000 (20:44 +0300)]
rrdtool: Read last update timestamp from database on startup
Instead of just re-updating all possible databases instantly on the
startup, read the last update timestamp from the database and do the
update with normal schedule.
This is useful if we just happen to restart the daemon and the last
update happened just a moment ago.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 30 Sep 2020 17:05:52 +0000 (20:05 +0300)]
rrdtool: Fix strncat() usage
As per strncat man page:
If src contains n or more bytes, strncat() writes n+1 bytes to
dest (n from src plus the terminating null byte). Therefore,
the size of dest must be at least strlen(dest)+n+1.
Therefore, we must ensure the destination buffere does not overflow is
src is large enough.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 25 Apr 2020 14:49:26 +0000 (17:49 +0300)]
rrdtool: Fix use of uninitialized buffers in string operations
The code is completely mess, referring to incorrect buffers as it is
calculating the size of the buffer. This wouldn't be problem as both
buffers are same length, but for some reason the code is also using
strlen() for something.
Fix up the mess.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 14 Mar 2020 07:52:01 +0000 (09:52 +0200)]
rrdtool: Take update time as we start updating db
Turns out commit 03aa50c69 didn't fix the drifting problem
completely. It is still possible that acquiring the data takes
arbitrarily long time and we end up inserting the data in the middle
of the database update interval, which will cause rrdtool to
interpolate the data point with previous or next entry.
Fix it by taking the timestamp before we start the update procedure
and using that time as we feed in the data.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 25 Aug 2019 12:30:58 +0000 (15:30 +0300)]
rrdtool: Stop update times from drifting
With rrdtool, database updates should rather take place evenly at the
beginning of each update interval. If it happens at random places
within the interval, rrdtool will try to interpolate the data to make
it "fit better" within the data points. This causes unexpected data
values to be generated into the database.
Make each updates happen consitently at the beginning of the window to
ensure the graphs contain as good raw samples as possible.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Thu, 22 Aug 2019 18:27:20 +0000 (21:27 +0300)]
onewire_parser: Implement simultaneous reading
DS18S20 sensors support simultaneous reading (not if parasitic
powered). This speeds up reading of multiple sensors massively, as the
lengthy conversion is done by all sensors at the same time.
But it does not work unless explicitly started. Also we don't want to
re-start the procedure after each sensor, as that would be as slow as
reading one by one.
Thus, we add a global shared timestamp that gets set once we read the
first sensor. After that, we have 10 second time to read the rest of
the sensors until we set it again. This should be more than enough
time to read large number of sensors in very short time doing only one
parallel conversion.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 20 Aug 2019 18:38:02 +0000 (21:38 +0300)]
onewire_parser: Fix "85" degree handling case
The current method of comparing the beginning of the raw string to
"85" will discard all measurements, even valid ones that are different
than strictly "85". This includes values such as 85.5, which can be
100% correct.
This also fixes a small memory leak that took place when we actually
got "85" from the sensor. If that happens, it is a failure, but the
*_read() function still had allocated the string for us. So it must be
freed.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Fri, 7 Sep 2018 18:16:27 +0000 (21:16 +0300)]
Silence gcc-8 snprintf overflow warnings
When compilint with gcc-8 and newer, we get warnings like:
/usr/include/bits/stdio2.h:64:10: note: ‘__builtin___snprintf_chk’ output between 11 and 2057 bytes into a destination of size 1024
return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
__bos (__s), __fmt, __va_arg_pack ());
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is because gcc thinks (correctly) that it is possible for the
resulting string to be larger than the buffer we ask snprintf to print
it. This is pretty bening warning, as we are not expecting to see this
kind of situations ever.
To work around the warning, add an error print for such cases when
actual overflow takes place. No actual memory overflow takes place
however as snprintf prevents this. But this also makes gcc happy as we
are now dealing the error case.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 29 Jan 2017 15:03:00 +0000 (17:03 +0200)]
onewire_parser: Fix crash caused by off by one memory allocation
The entries in sensor list always start with either server address or
mount point, which is not counted as a sensor. If we want to count the
actual number of sensors, we obviously need to add one more to the
last index number, otherwise we get one too small number for sensors
and allocate too little of memory for the parser state.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Thu, 26 Jan 2017 18:36:32 +0000 (20:36 +0200)]
onewire_parser: Increase glitch detection retry count to 4
In order to detect glitches properly, we might need to do more than
two reads, especially on noisy lines. Double the amount of reads we do
so that we have better chance of removing glitches from the data.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Thu, 26 Jan 2017 18:28:32 +0000 (20:28 +0200)]
onewire_parser: Make step changes pass glitch detection
The glitch detection code does not recognize step changes at
all. Currently it re-reads the temperature values until maximum glitch
threshold (two reads) and then just uses the last data. This is not
ideal on noisy lines as the last data might be a glitch itself.
Improve the logic so that after considering for a glitch, we compare
two consequent readings and see if the delta between those two is less
than the glitch threshold. If it is less, we consider the data
good. This is needed in case the temperature is changing quickly and
we don't get two identical values, but almost same.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 14 Jan 2017 15:39:14 +0000 (17:39 +0200)]
Improve mutex debugging prints
When lock contention is detected, print also the name of the thread
holding the lock and also where the lock was tried to acquire.
After the lock is finally acquired after the contention, add a new
print there as well to indicate that the lock is now acquired and the
thread will continue.
This adds overhead to the mutex operations, but that shouldn't be an
issue yet. If it becomes too slow, an option can be added later to
remove the prints and the overhead altogether. Now we are more happy
about the improved verbosity of the locking operations.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 14 Jan 2017 15:32:40 +0000 (17:32 +0200)]
worker_thread: Drop process priority when executing low priority work
When executing low priority work, it is a good idea to nice() the
process priority as well so that the executing of the work is truly
run as a low priority work. This decreases the interference the work
might have on the rest of the system.
To achieve this, the work executing loop is rewritten. Instead of
having a loop that always tries to execute the highest priority work,
separate the low priority and high priority handling in separate
loops. This simplifies the logic quite a bit. Furthermore, there is
really no need to convert a low priority worker back to executing high
priority work. When queuing work, a new worker thread is always
spawned whenever a high priority work is queued. Thus, there is no
really any chance a low priority worker could pick up any high
priority work once it has finished low priority work.
Now we no longer can do that as it is not possible to rise a thread
priority back up once it lowered its priority.
The executing logic of the worker thread is thus much simpler now.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 15 Nov 2016 17:44:47 +0000 (19:44 +0200)]
Fix possible buffer overrun due to incorrect strncat length argument
The strncat usage assumed that the length argument to strncat
indicates the length of the destination buffer. That is how strlcat
works. The length argument for strncat instead describes the maximum
number of characters to copy from the source buffer.
To make the call sites work correctly when we want to avoid
overflowing the destination buffer, we need to subtract also the
current length of the destination buffer string.
This also cures possible overflow issues with any of the strncat use
sites.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 1 Nov 2016 19:02:00 +0000 (21:02 +0200)]
onewire_parser: Ensure proper strng NULL handling
strncpy() call length argument must be one less than the actual buffer
length. The final byte in buffer must be NULL, in case strncpy does
not fill it with NULL itself.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 1 Nov 2016 18:56:54 +0000 (20:56 +0200)]
onewire_parser: Use larger string buffers in make_uncached()
The strings used for temperature sensor paths might be longer than 32
bytes. If that is the case, we might truncate the actual output
string, thus making temperature sensors inaccessible in case we might
need to read an uncached data from it. This become visible after the
glitch detection code started to make frequent re-reads from sensors.
Fix it by allocating 1024 bytes for the strings instead of 32
bytes. This should be more than enough.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 1 Nov 2016 18:54:49 +0000 (20:54 +0200)]
mkcompile_h: Use mv instead of cp
Once temporary version files are done, replace the original with the
temp file instead of copying over it. We don't need the temporary copy
any more so we can move it over the target file.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Fri, 28 Oct 2016 17:38:43 +0000 (20:38 +0300)]
onewire_parser: Implement glitch removal
Some times in noisy 1wire networks there might be random bit flips
during read, which slip through even from the CRC check done in
line. This might lead to errorneous readings that cause spikes to
graphs.
Implement a relatively straight forward logic for avoiding the
glitches. Here we keep a track of 20 delta values between samples and
the latest reading (so that we can calculate the new delta). If the
latest delta is significantly larger than biggest known delta within
the sample history, we will retry reading the sensor. If we get
exactly same value back we trust the reading. If the value keeps on
changing or we get too many failures, we get what we can.
This should allow recovery for most simple transient glitches.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 26 Oct 2016 17:55:08 +0000 (20:55 +0300)]
Allow parsers to store private data to databases
Pass a pointer to parser function where the parser can store one
pointer worth of its private state. This can be useful in different
ways whenever a parser needs to store some information between parser
calls.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 26 Oct 2016 17:19:12 +0000 (20:19 +0300)]
onewire_parser: Fix double free corruption
If there is a failure while reading a sensor value, the pointer
pointing to the destination buffer is undefined. In that case we might
be referring to a previously freed pointer and possibly corrupting
data somewhere else. The heap area might also be already allocated for
something else and freeing it here, making the heap area available to
yet another allocation and further corruptions. This leads to random
and hard to reproduce crashes in various unrelated components.
Fix the issue by initializing the pointer value with NULL. Also avoid
using it when we expect it to be NULL after a failure. Also we are not
passing the NULL pointer to free(), even though calling free on NULL
is bening.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 24 Jul 2016 12:39:45 +0000 (15:39 +0300)]
onewire_parser: Fix memory leak when retrying OWNET_read()
If it happens that read fails for any reason and the code retries
reading, the previous read buffer is not freed properly. This causes a
minor (26 byte) memory leak on each retry. Add a free() call to avoid
the leak.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 24 Jul 2016 11:12:37 +0000 (14:12 +0300)]
Revert "onewire_parser: Keep server handle open at all time"
This was not a very good workaround after all. It appears that on some
versions of ownet the client code leaks sockets even if it reuses the
server connection all the time. No point in having workaround like
this.
Timo Kokkonen [Mon, 11 Jul 2016 11:56:48 +0000 (14:56 +0300)]
onewire_parser: Keep server handle open at all time
On libowfs 2.7 there appear to be a problem when releasing the client
resources. At least sockets are left open every now and then. There is
no such problem with 3.1 version.
To resolve the issue with older client library, keep the serve handle
open at all time. Release it only when there is a change with the
server address, which normally does not happen ever.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Mon, 11 Jul 2016 11:53:34 +0000 (14:53 +0300)]
onewire_parser: Avoid variable name collision
Counter 'i' is already used by the outer while loop. Re-declaring the
varialbe would not be a problem unless we had also the print below
that expects to use the variable from the outer while loop.
Rename the inner variable to avoid conflict.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 9 Jul 2016 18:10:04 +0000 (21:10 +0300)]
rrdtool: queue post_draw_cmd to be after images
The purpose of the post_draw_cmd is to execute something after the
rrdtool has drawn the images. Right now it executes the command
actually before drawing the images, as the command gets executed from
the high priority work that queued the image drawings.
This is not what we want. Therefore queue the post_draw_cmd to the
same low priority queue where the images are put. This guarantees that
the command does not begin too soon. On uniprocessors it is also
guaranteed that the command does not start executing before all of the
images are completed.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 9 Jul 2016 17:18:13 +0000 (20:18 +0300)]
process: Handle threads dynamically
Instead of hard coding thread number on start up, handle them
dynamically as needed.
The max_jobs variable determines how many workers there may be
running. However, there are also high priority works that should be
executed even though thread count is already maxed. Thus we start a
new worker whenever a high priority work is queued.
As soon as work is finished and there are no more work to do or there
are too many threads to do low priority work, threads will exit.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 9 Jul 2016 11:09:40 +0000 (14:09 +0300)]
process: Remove signal handler completely
There is no need for this at all any more. This was only used for
reaping the childs on the fork based concurrency management. Now that
threads are doing forking on the same namespace, threads that are
expecting signals might get confused when the main thread handles the
sigchld signal on behalf of the thread. Remove the handler completely
to avoid any confusion.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 9 Jul 2016 06:50:07 +0000 (09:50 +0300)]
process: Make run() execute synchronously
Currently there are no need for running the process asynchronously on
background, so we might just make it run synchronously instead. This
simplifies the calling convention and also removes completely the need
to fork and possibly race in glibc functions.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sat, 9 Jul 2016 06:44:51 +0000 (09:44 +0300)]
Convert rrdtool to use work queues instead of forks
Switch to the work queue model. Change is somewhat trivial. The
database update timestamp update is now done before the actual
update. We don't have a mean to distinguis from the fact that we have
initiated an update but not done it yet, so we mark it as "done" and
then queue the update. Without this change the main thread will keep
on queueing the update indefinitely as the worker thread takes too
long to actually mark the update as done.
To ensure the queues work properly, we also need to disable sigchld
handler from the process controller so that it does not get confused
when threads are reaping the childs them selves. This is different
from the previous model where each process required to handle their
own child reaping after fork(). Now we are sharing the signals with
the parent. This is a major simplification on how this works.
The magic sleep on job control init can be removed as well as there
are less forks going on now.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Fri, 8 Jul 2016 19:44:39 +0000 (22:44 +0300)]
process.c: Introduce work queues
Work queues are a way to execute function calls asynchronously. Each
work is placed on a fifo queue where worker threads pick them up one
by one.
Priority levels are also supported, right now two levels are
available. All high priority works are executed before workers pick up
low priority works.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Fri, 8 Jul 2016 19:17:09 +0000 (22:17 +0300)]
debug: Print thread name on debug prints
Many different threads might be calling the same functions
concurrently. Add the name of the thread so that it becomes more
obvious which thread is emitting the given print.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Fri, 8 Jul 2016 19:42:42 +0000 (22:42 +0300)]
process: Add debug wrappers for pthread mutex operations
This is the beginning of the work to convert the concurrency model
from fork-bomb based model to pthreads.
To get started, add a wrapper functions to handle locking. Namely,
these wrappers will log where the lock was acquired and will print out
who is holding it in case of contention.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 28 Oct 2014 19:25:40 +0000 (21:25 +0200)]
process: run: Fix bug with multi-line prints
Once newlines are replaces with NULLs for printing, we must start the
next line after the NULL. Without this all extra printout lines are
zero in length.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 28 Oct 2014 19:16:21 +0000 (21:16 +0200)]
rrdtool: Add support for post_draw_cmd
Just like pre_draw_cmd, post_draw_cmd is a command that gets to be
executed after graphs are drawn. This is useful for example
transferring generated graphs into final location once rendering is
complete.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 26 Oct 2014 15:12:34 +0000 (17:12 +0200)]
rrdtool.c: Null terminate process argument list in add_arg
add_arg macro was not null teriminating argument list. Null
termination was left as an responsibility for the caller. This is
error prone as this requirement is not clear.
Add null termination in the add_arg macro so that caller does not need
to care about it.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 9 Sep 2014 19:17:37 +0000 (22:17 +0300)]
process.c: Reduce maximum number of parallel jobs on UP machines
We generally don't see sinlge-cpu systems any more in other than
embedded devices, where chances of being very low in RAM is a very
real possibility. If the maximum number of parallel jobs is too large
there, is is quite possible that we run out of memory on such systems.
Reducing the number of jobs makes such devies much less likely to
exceed their memory capacity.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 20 Aug 2014 19:07:05 +0000 (22:07 +0300)]
onewire_parser: Make sensor readin more robust on noisy 1-wire networks
On noisy environment there may be transient read failures with some
1-wire sensors. The sensors might "come and go" and also read results
might fail with "85 degree results".
Try to work around the issue by retrying a sensor read with uncached
data. This helps in cases where there was a error during 1-wire scan
and some sensors were not detected. This also helps when sensor
responds but reading the result produces "85" degree result.
Reading should be significantly more reliable with this change.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 6 Jan 2013 13:49:14 +0000 (15:49 +0200)]
Add support for data logging
Add new configuration parameters "logfile" and
"logfile_timestamp_fmt". Both of them are fed through strftime. The
first one is used for creating log file name, the second one is the
time stamp format in the beginning of each line.
Neither of the options are mandatory. If no file name is given, no
logging will be done. If no time stamp format is given, default of
"%Y.%m.%d %H:%M " is used.
Each line in the log file contains all the data elements given to
rrdtool during updating the database. The format is altered
slightly. The beginning of the line has got the time stamp, the
beginning "N:" part is removed and all ':' characters are replaced
with white space. Each entry is padded up to 12 characters with white
space in order to make the log file entries nicely human readable.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Fri, 30 Nov 2012 16:13:53 +0000 (18:13 +0200)]
Makefile: Adjust default targets
The "all" target will now build all parsers in addition to the main
executable. The "default" target builds only the main executable, thus
default setup will not have libownet build requirement.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 20 Nov 2012 20:08:07 +0000 (22:08 +0200)]
Makefile: Fix typo in object list
The ONEWIRE_PARSER_OBJS was written incorrectly. This causes ALL_OBJS
list to miss the parser object, which in turn causes the dependency
file to be missing from the ALL_DEBS list. That was causing make to
not detect that onewire_parser.c depends on version.h, and not rebuild
the library when version.h was changed.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Mon, 19 Nov 2012 20:12:09 +0000 (22:12 +0200)]
onewire_parser: Convert to a plugin
Onewire parser is the only piece of code here that depends on libownet
and requires linking against it. This is cumbersome whenever we wish
to build rrdd and we are not interested in having support for onewire
at all.
Converting the parser to a plugin solves the issue: We can now build
main rrdd without libownet dependency. Even if we have built the
onewire parser, we don't need to load it unless we are actually using
it.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Mon, 19 Nov 2012 19:59:00 +0000 (21:59 +0200)]
parser: Try loading a parser from a plugin in case no parser is found
When a parser is queried and one is not found within the ones that we
have already registered, try loading a plugin with that name. If
parser plugins are available, the parser might be available after the
plugin has been loaded.
The project is now fully compatible with having parsers as plugins.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Mon, 19 Nov 2012 19:45:39 +0000 (21:45 +0200)]
plugin_manager: Load parser plugins by name
A parser plugin is a plugin that has name "%s_parser.so", where "%s"
is the name of the parser. Implement a function that will try to find
a parser plugin by a name. If we can't find such plugin from the
global library path, try some other places as well, such as current
working directory and executable path.
In order to know the executable path, the plugin manager needs to be
initialized with the executable path. We are now doing it in the
beginning of the main function.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Mon, 19 Nov 2012 18:43:52 +0000 (20:43 +0200)]
Parsers: Implement framework for registering and querying parsers
This framework can be used to register parsers and get parsers by a
string name. This allows all need for hard coding string and parser
function names together.
All existing parsers are converted to use the new framework.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Mon, 19 Nov 2012 16:48:38 +0000 (18:48 +0200)]
Remove database.h
Instead of having a database built in the software, have one being
provided with a config file instead. Keeping the database header file
is becoming a maintenance burden.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Mon, 19 Nov 2012 15:31:18 +0000 (17:31 +0200)]
Introduce plugin manager
Plugin manager can load plugins at run time from libraries. It expects
the plugin to contain one symbol, which contains plugin_info structure
defined in plugin.h file. The .init function is executed from within
the structure.
Add -rdynamic compiler flag in order to make the existing symbols
become visible to the dynamically loaded libraries.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 28 Oct 2012 16:12:08 +0000 (18:12 +0200)]
jobcontrol: Fork bomb prevental
If something happens that prevents existing jobs from terminating, it
is generally bad practice to keep on forking more processes
indefinitely. To alleviate the problem, implement a somewhat trivial
process limitation feature. This will define a limit of maximum number
for the processes pending execution. If the number exceeds the global
limit, new processes are to be terminate itself immediately.
This however prevents only "limited forks" for creating too many
processes for waiting execution. Normal forks still have no
limit. This should still make the system to stay alive longer during
situations where something prevents (temporarily) executing processes
to finish in time.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 28 Oct 2012 10:45:01 +0000 (12:45 +0200)]
rrd images: Make the timestamp string configurable
Make it possible to replace the "Last update ..." string with
something that fits better for the needs of the end user. The string
is passed through strftime so one can use arbitrary timestamp formats.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Thu, 11 Oct 2012 18:30:20 +0000 (21:30 +0300)]
process: Implement event handling infrastructure
Implement proper event registration infrastucture for handling the
events coming in through the file descriptors. This makes it possible
to expand the polling code to arbitrary number of file descriptors.
All current users of the epoll have been converted to the new
infra. Some functions were moved higher in the file in order to make
it possible to reference them in the init_jobcontrol() function.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Mon, 1 Oct 2012 14:45:52 +0000 (17:45 +0300)]
process: request_fork: Ensure function parameter is always signed
Some architectures, such as armv6, treat char as an unsigned integer
instead of signed. Basically it makes no sense to use char type with
the variable. When it is changed to an int there is no confusion about
the type and it will work better with all architectures.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 17 Jul 2012 11:26:22 +0000 (14:26 +0300)]
Merge scheduler.c into rrdtool.c
The schedule calculations were all very much dependent on the rrdtool
code and not something independent. It doesn't make sense to keep it
separate from the rrdtool code.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Mon, 2 Jul 2012 18:55:24 +0000 (21:55 +0300)]
onewire_parser: Add support for temperature offsets
Some temperature sensors may have a significant systematic error
offset on their readings. This patch makes it possible to define an
offset value that is added to the raw value before the result is
stored in the rrd database.
From now on it is assumed that each onewire path never contains white
space. If there is a white space within the server path, it is
expected that each string that is separated with white space contains
an option that is used to control the readout of the values returned
or available from the server.
At the moment only one option is supported; temperature offset. The
option string must contain "offset=" following a decimal number. The
offset value is added to the number read from the server.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Thu, 28 Jun 2012 07:55:41 +0000 (10:55 +0300)]
scheduler: Give the name of the next db to be updated
This is somewhat useful in debugging as the user will be able to know
which database will be updated next, so he can anticipate what kind of
output there is to be expected soon.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 27 Jun 2012 19:00:50 +0000 (22:00 +0300)]
script_parser: Fix fread to return number of bytes read
From fread man page:
The function fread() reads nmemb elements of data, each size bytes
long, from the stream pointed to by stream, storing them at the
location given by ptr.
If size is something else than 1, the return value is not number of
bytes read. The intention here is to have it return number of bytes so
that the debug print can print it correctly.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 27 Jun 2012 15:35:50 +0000 (18:35 +0300)]
process: Print process output through the debug macros
As we are appending our own prefixes to all process prints, we might
aswell use the normal debug macros to unify the prints with the rest
of the debug output.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 27 Jun 2012 15:29:14 +0000 (18:29 +0300)]
parser: Remove old debug print
All debug prints should use the pr_* variants instead of printf
directly. This one print in question is not very useful, so it can be
removed altogether.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Wed, 27 Jun 2012 15:18:19 +0000 (18:18 +0300)]
process: Fix bug with child processes stuck forever when parent segfaults
After a fork() the child needs to close the unused end of the process
control pipes. Failing to do so will prevent EOF and EPIPE signals
from being delivered correctly back to the child when the parent
closes its end of the pipe.
Furthermore, the code's assumption about the behaviour of the pipe
with one writer and multiple readers was wrong. When there are
multiple readers, only one will be woken up and the rest will be left
blocking the read in case there was nothing left to read. If read
returns zero, that means EOF, eg. the job control parent has closed
its end of the pipe.
These fixes applied there should be no more problem with a lot of
child processes stuck in waiting the parent to give them a permission
to go. They will either fail with SIGPIPE when they request the
permission to run or they will read zero bytes from the pipe and
continue executing instantly.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 26 Jun 2012 13:40:58 +0000 (16:40 +0300)]
debug: Reintroduce message colouring
When the debug macros were refactored into a separate function, all
messages were printed with green color. Change it to use the desired
color instead.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Tue, 26 Jun 2012 13:33:14 +0000 (16:33 +0300)]
debug.c: Remove indentation from the prints
The child-depth indentation is not really that useful. In fact, it
makes actually harder to read similar debug print lines with different
indentation. The numerical level of the child generation gives
sufficient information.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>
Timo Kokkonen [Sun, 24 Jun 2012 19:59:23 +0000 (22:59 +0300)]
debug: Rework debugging
Instead of having a huge debug macro that gets expanded every time a
debug print is needed, separate the debugging facilities into
debu.c. This will reduce excess code duplication a lot.
A concept of debugging level is also introduced. This makes it
possible to add debug prints with different priority level and have
some of the prints to be silenced when user is not interested about
them.
Signed-off-by: Timo Kokkonen <timo.t.kokkonen@iki.fi>