About me
Little syslog-ng configurator application for Android
As you know I like syslog-ng but also my hobby is Android programming.
Because I wanted to practice (and develop my Android skills), I’ve
just written a little interesting Android application related to
syslog-ng performance tuning.
It can calculate the tuning related settings like log_iw_size(),
log_fetch_limit(), log_fifo_size() and so on, based on the given source
specific parameters. Furthermore it can generate an example
configuration that can be saved or sent via email.
So it was absolutely just my hobby project related to Android, but after
I showed it to a few people interested in syslog, I received positive
feedbacks. After I decided to share it with everybody taking into
consideration that I already have an Android developer account.
I hope it will be useful, at least you can play with the numbers and
results, but of course this is not a serious program just as I said my
hobby project.
You can download it from Android Market (and you can find more details
here): syslog-ng Configurator
Here are a few screenshots:
Jenkins vs JUnit XML format
Recently, I wanted to add the result of our test system to the Jenkins, especially to the Publish JUnit Test Report plugin to analyse and process it.
Unfortunately, the output of test system was a custom output (plain text file) and not created by any Java program but it contained passed, failed/errord testcases thus I assumed it can match to the XML format.
So the question was the following:
How to convert this custom output to JUnit XML format that Jenkins can process?
After I’ve spent a lot of time with Google to find the way, I decided to share my experience with you.
First of all, I wanted the get the XSD to the XML format. It was not easy, and I realized that basically the XML output is not created by JUnit (Junit.org) but Apache Ant does it.
Secondly, I was surprised when I realized that JUnit plugin uses different XSD that the official Ant.
Here is the Ant XSD
and here is what JUnit plugin uses: JUnit XSD
Finally, I started to use this XSD as base of my XML and I could convert my plain text result to the proper XML format.
How does an XML look that Jenkins can parse?
Here is an example:
<?xml version=”1.0″ encoding=”utf-8″?>
<testsuites errors=”1″ failures=”1″ tests=”3″ time=”45″>
<testsuite errors=”1″ failures=”1″ hostname=”localhost” id=”0″ name=”base_test_1″ package=”testdb” tests=”3″ timestamp=”2012-11-15T01:02:29″>
<properties>
<property name=”assert-passed” value=”1″/>
</properties>
<testcase classname=”testdb.directory” name=”001-passed-test” time=”10″/>
<testcase classname=”testdb.directory” name=”002-failed-test” time=”20″>
<failure message=”Assertion FAILED: some failed assert” type=”failure”> the output of the testcase
</failure></testcase>
<testcase classname=”package.directory” name=”003-errord-test” time=”15″>
<error message=”Assertion ERROR: some error assert” type=”error”> the output of the testcase</error>
</testcase>
</testsuite>
</testsuites>
I think this example explains everything. Basically, JUnit parser is not too smart, so if you add an “error” lines to the testcase it will be marked as errord, if “failure” then it will be marked as failed if you don’t add these lines, it will be marked as passed. The property is an optional thing, I’ve just added it because we use some internal state for the testcase results.
That’s all.
And here is the result in a picture:
Start screen after ran
Problems with Jenkins
No difference between errors and fails
After I’ve created the proper XML, found some interesting behaviour in Jenkins. Firstly, it does not distinguish the errord and failed testcases, it shows all of them as failed. It’s not good because it can mask the root cause of the problem. The good news that they already know it:bug-4951
No difference between errors and failes…
Package value is always root
Secondly, it does not show the value of package field as package, but the content of the classname, everything before the last dot. Thus if you want to use your real package name, just fill the “classname” field in the following form: package.class
Package as root
Package as classname
[.] More
Slow Android with HTC Wildfire
I have an HTC Wildfire phone with Android 2.2 but unfortunately it became very-very slow. To tell the truth it was already a slow system when I bought it newly (…) but it had more or less acceptable speed.
For now, it has become an almost unusable mobile due to performance problems (it is very slow, sometimes it has hard lag or just freezing) and I couldn’t find the reason. I asked some of my friends who also use Wildfire and they also complained about the slow performance and lag.
It has a lot of irritating symptoms, for example loading the call-list is very slow and when I try to call someone clicking on his name the mobile starts to call wrong person because the call-list moved away between clicking and calling… Or when I want to read an SMS it take a lot of time while it is able to open the SMS store… So like in the age of the old Windows, after you switched on your computer you could go to drink a cup of coffee…
I removed all of not necessary applications (however I didn’t install too many applications just a few one to my phone) and installed a task killer application to kill the not required processes, because I assumed that’s the reason for this behavior, but it didn’t help. I was angry, because I really didn’t want to buy a new mobile because I like this mobile.
After reading a tons of forums I realized that the problem is the HTC Sense/HTC ROM. It starts a lot of applications that are not needed for me (e.g.: some kind of stock application and so on) and you can’t remove it without root permission. And it seems that the HTC Sense layer is very slow and buggy at least on my mobile and you can’t remove or turn it off.
I have decided to remove HTC Android from my mobile, and install Android v2.3 with CyanogenMod 7. This was a very good choice, my Wildfire has become incredible fast after I upgraded it to Android v2.3. It works like a charm, no more lagging and no more waiting when I want to do something. Sometimes I have a feeling that I’ve bought a new mobile because so big the difference between the old system and the new one.
How can you upgrade your Wildfire?
Unfortunately, you can’t do that officially, because HTC does not allow you to upgrade to Android v2.3 and you can’t turn of HTC sense without root permission.
Finally, I’ve found a very good description about the upgrade. This description has only one missing step, removing S-ON flag from your HTC thus I described how can you do that.
Don’t forget, you can do that only at your own risk, you can lose the warranty of your mobile or if you don’t understand and don’t follow the steps your mobile can become a brick.
However, I didn’t encounter any problems when I followed the steps they are quite easy.
Removing S-ON
First of all, you need to turn S-ON setting to S-OFF. This means the security level of your HTC, if it is set to ON, you won’t be able to get root permission and replace the ROM.
For more details about S-ON/OFF
To turn it off, you need download and install Revolutionary software, usb driver for android and of course you need your HTC, and an USB cable for your mobile. Just download the usb driver and install it, then download the Revolutionary software, start it, and plug your HTC to the computer. If you didn’t enable debugging mode for your HTC, enable it before plugging: go to Settings > Applications > Development and enable USB debugging.
The command line application will detect your mobile and do the required steps (it may reboot your phone). You need select your Windows and HTC version on the revolutionary web page after downloaded the software, and add the serial number provided by the Revolutionary when it detects your mobile. If everything is fine, the web site will give you a key that needs for revolutionary and you need to wait while the program is finishing its job.
Now, your mobile is ready to finish the remain steps. The only thing is left you need to follow this description, and your HTC will be ready within 20 minutes. Based on my experience it is worth the time, as I’ve mentioned, my mobile seems at least twice faster than it was anytime before (and with v2.3 you can take screenshots by default 
Here are a few screenshots about my “new” HTC with Apple theme(sorry for Hungarian labels):
Zero Message Loss with syslog-ng: Promise or Reality?
Hi,
syslog-ng PE 4.2 contains a new protocol, RLTP. Perhaps you’ve already read about it in our marketing materials, but you would like to know what it is and why you need it.
RLTP
First of all, why is RLTP needed if you are already using TCP, a “reliable stream delivery service that guarantees that all bytes received are the identical bytes sent and in the correct order” (source: wiki) and ensures data stream integrity with acknowledgements?
The answer is simple. TCP does not guarantee that the destination application receives the packet; it only guarantees that the destination operating system receives the packet. In fact, when the specified module of the kernel receives a TCP packet it will answer with an acknowledgment message. However the destination application may not have received the packet yet. In the event the destination application stops/crashes, the operating system crashes, or something evil happens after the operating system has acknowledged receipt of the packet but before the destination application has received the packet this can cause message loss. In this scenario, the originating application will assume the everything is fine but the destination application never receive the message…
Also, even if the application has received the message, usually there is no guarantee that it could process it. For example, if we speak about syslog-ng, after it receives a message it tries to write the message to disk or forward to the destination. But if this operation is unsuccesful (e.g.: the disk becomes full) and you stop the syslog-ng, this message can be lost and the originating side won’t be able to detect it (because the message has been received normally).
How can you avoid message loss with syslog-ng? The answer is Reliable Log Transfer Protocol (RLTP).
In fact, this is an application layer acknowledgment protocol. The point is that when syslog-ng forwards messages via RLTP and the destination syslog-ng application can process them, the destination syslog-ng sends an acknowledgment message confirming successful transmission. If the sender does not receive an acknowledgment message or this message indicates that the destination syslog-ng could not receive/process all of the sent messages, the sender will know that problems occurred with the transmission and will resend the missing messages.
When does the syslog-ng destination application send an acknowledgment message? In the event that the destination syslog-ng can write the messages to the hard disk, store them in the disk buffer or, in the case of a relay, when it receives acknowledgment message from the remote syslog-ng. So it’s a chain, when the last syslog-ng in the chain processed the message (e.g.:the message has been written to disk), it sends an acknowledgment message and when the previous syslog-ng receives acknowledgement, it will send back an acknowledgment message to the previous syslog-ng and so on while the acknowledgment message reaches the first syslog-ng. If the chain is broken somewhere, the first syslog-ng won’t receive the acknowledgment message and will repeat the messages.
For more deatils: RLTP
Reliable disk-based buffering
Well, I spoke about the server side, but what will happen if someone kills the originating syslog-ng?
This could be a a problem because syslog-ng uses memory buffers by default to store messages (configured via log_fifo_size() option) and the disk-buffer also has memory buffers. This is necessary because memory is faster than the hard disk. If someone kills syslog-ng (e.g.: with SIGKILL signal), the content of the memory buffers will be lost. If you stop syslog-ng and it cannot write all of the messages from memory buffers (for example, if the destination becomes unavailable), the messages will be lost.
The solution is a reliable disk-based buffering; it is a new feature of syslog-ng PE v4.2.
Unlike the standard disk-based buffering in syslog-ng, the new reliable disk-based buffering disables all memory buffers excepting only one (specified via mem-buf-size() options), but this is a special memory buffer: syslog-ng stores all content of this memory buffer on disk, too. (this special memory buffer is required due to technical reasons). When a message comes in, it will goes into the disk-buffer and it is absolutely stored on the hard disk. If someone stops/kills syslog-ng, no problem, the messages are stored on the disk. When you start syslog-ng again, it will find the messages in the disk-buffer and continue its work.
syslog-ng will remove a message from the reliable disk-buffer, when the destination has received the message. For example, if you are using RLTP, when the acknowledgement message receives. No more message loss.
In a word, when you are using RLTP protocol everywhere and reliable disk-based buffering, you can achieve zero message loss.
Of course, because the server and the client can work with different speed, you must enable flow-control flag on the log path to control it (but this is not a new function in v4.2). flow-control means that syslog-ng will stop reading messages from the sources of this log statement if the destinations are not able to process the messages at the required speed.
For more details: Reliable disk-based buffering
Configuration
Finally, here is a simple example configuration for the client and the server. The client receives the messages from file and forward them via RLTP using reliable disk-based buffering and the server receives the messages via RLTP and stores them in a file.
Client configuration:
@version: 4.2
options {
threaded(yes);
time_reopen(10);
};
source s_file{file(“/var/log/inputlog.log”);};
destination d_syslog_rltp {
syslog(
“127.0.0.1″
port(“4444″)
transport(rltp)
disk-buffer(mem-buf-size(200000) disk-buf-size(2000000) reliable(yes) ));
};
log{
source(s_file);
destination(d_syslog_rltp);
flags(flow-control);
};
Server configuration:
[.] More@version: 4.2
options {
};
source s_syslog_rltp {
syslog(
transport(rltp)
port(“4444″)
);
};
destination d_messages { file(“/tmp/test.log”); };
log {
source(s_syslog_rltp);
destination(d_messages);
flags(flow-control);
};
Digital astrophotography: beginning
Update: this post is obsolated, the newest version can be found here
Although my hobby is the amateur astronomy and I’ve been watching the sky for a few years, I began to take photos about the sky only a few months ago.
After I’ve got some experiences in this topic, I decided to write a blog, because it was not easy to get the necessary information how to do it or what type of pictures I can take with different cameras and devices. To tell the truth, I didn’t want to spend a lot of money while I didn’t know it will worth.
Just to clarify, I wont introduce the professional digital astrophotography or all parts of this theme but try to introduce what kind of problems a beginner can run into and how nice photos he can take with different cameras.
So, it is an astrophotography description from a beginner to beginners. Because I am also a beginner, please forgive me if I make some mistake.
Before starting the post, let me draw your attention that you shouldn’t expect such a perfect photos as Hubble Space Telescope does, but don’t forget: creating own photos can be so magical and I think it’s a very good thing in the life.
However, if you don’t want to take photos, you can watch the sky looking into the telescope with your naked eyes, and it also can be very amazing. Unable to write the experiences of the first look when someone discovers the moons of the Jupiter or the rings of the Saturn, and usually everyone shouts: “wow, it’s moving…”
In the furthermore, I assume that you already have some astronomy knowledge and tried to use a telescope, if not, you should learn the basics before dealing with astrophotography.
And now, here is a list that you need for astrophotography:
The telescope
To choose the best telescope is always a hard decision because there are a lot of different telescope types, some of them are suited to watch the Moon and the planets, and with other ones you can observe the deep-sky objects. In my case, I am using a Skywatcher 150/750 Newton reflector telescope, the numbers means that the diameter of the telescope is 15 cm, and the focus length is 75 cm.
Generally, this is an entry-level class for astrophotography, the telescopes under this class are not capable to make good photos.
However, choosing the telescope is always based on compromises because for bigger telescopes need bigger place and it’s difficult to move them but you can use them to take better photos. Usually the ones having shorter focus are offered for looking deep-sky objects, and the ones having longer focus are offered for looking planets but as you will see, this is just a recommendation.
Here are a few photos about my telescope:
The mount
First of all, what you have to know on it: the mount is at least so important as the telescope, in the astrophotography. When you use your telescope at high magnification (200-300x), a little wind can cause bigger excursion in the telescope, and it makes impossible to take photos. In my case, I have an EQ-3 mount, with all equipments its weigh is around 20 kg. This is an basic mount for astrophotography, but the bigger ones can be very heavy and if the observation point is far from your location or you need to move the telescope more than 10-20 meters, it can be very exhausting.
However, it’s recommended to use better mount, but this is also a hard decision, because you need to choose between the easiest moving and the massive, stable mount. Anyway, using more stable mount, you can take better photos.
The driving
Now, you have a telescope and a mount, but to take photos you need a motor to control the telescope. Due to the Earth rotation, the objects are also moving on the sky and the telescope needs to follow them. For example, the Jupiter will go out from the field of view during 20 sec at 200x magnification. The easiest way is to use a RA driving motor, but if you want to take photos about the deep-sky objects, you should use a better system, like computerized auto-guiding (GOTO) systems. For now, I am using only RA driving.
The following video will illustrate how quickly Jupiter is moving without driving:
The Jupiter without driving
The camera
Basically, you need to know two things:
The CCD cameras are used to make films about the planets or the Moon, and a special computer software will create the photos from this film (which contains hundreds of frames).
For taking photos about deep-sky objects, DSLR cameras are used with long exposition time because these objects are very pale, and the camera needs to collect as many light as possible. Due to the long exposition time, a precision driving is also needed and a massive mount. However, the DSLR cameras are very expensive thus I recommend to use it if you already have one, or you already know the basics of astrophotography. Because I don’t have DSLR camera, I wont introduce it in this blog.
I will only take photos with compact camera and with CCD camera.
Compact camera
Most of the people would like to take photos with its compact camera, no doubt, it’s the simplest solution, but the compacts are not really suitable for this purpose. However, you can take very nice photos about the Moon.
In the furthermore, I will compare the photos taken by a compact camera to a CCD camera.
First photos
I am using the following equipments: Skywatcher 150/750 Newton telescope, oculars, barlows, EQ-3 mount, RA driving, Scopium planet camera, a compact camera (Samsung s760) and a notebook for storing the films and photos.
Firstly, let’s see what kind of photos you can expect, if you take same photos with a compact (the target is the Moon, in order at 80x, 160x, 240x magnification):
The above photos succeed very good, taking into consideration that the Moon is an ideal target for compacts, because it has good brightness and no apparent rotation. Based on my experiences, using lower shutter speed (1/30, 1/45) I could take better photos.
But what will happen if I try to take photos about the Jupiter?
The photos would be blurry, and not too detailed. That’s the disadvantage of the compacts. You cannot remove the lens of the photo camera thus you need to use also an ocular in the telescope and thats the reason why you cannot make very good photos. It does not matter in case of Moon, but it is in case of planets or the deep-sky. Tuned the compact settings my best picture was the next:
In the next step, let’s see what kind of photos can we take using the Scopium planet camera. Actually, this is a special CCD camera with built-in IR filter and it can join into the telescop easily. Ocular is not required, and there is no lens for the camera thus you can minimize the lens errors. The maximum resolution is 640×480 and the maximum frame rate is 30 fps. This camera can record the video in raw uncompressed format, that means that 1 frame will be ~1 MB thus a 10 seconds length video will use 150-300 MB space on the disk and you have to make a dozen video in a night.
To take a photo in proper resolution, I need a barlow lens, it will increase the focus of the telescope, after using it, the the focus will be 225 cm. Because the size of the CCD chip is usually very small, you also need to use a barlow lens. As I mentioned it earlier, it’s very important to drive the telescope because the sky objects are moving very quickly. In the following, I will show a few photos about the Jupiter without driving:
I think these photos are better then the previously ones created by compact, but they are not really detailed.
And now, here are the bests. I created the following photos with Scopium camera, using 3x barlow lens and with RA driving:
I think the above photos tell everything. The difference is amazing compared to the compact. Just a notice:
You can see the similar picture in this telescope even if you look it into with naked eyes, so the Jupiter will be as detailed as you can see it on the above photos (it’s good to know, because a lot of deep-sky objects are so beautiful only on the photos due to the long exposition time).
Making deep-sky photos with compact
As mentioned earlier, DSLR cameras are the best for this purpose, but I tried to use my compact to take photos about deep-sky object. However, the pictures would be noisy and not too detailed, but I expected worse pictures after reading a tons of forum about it. Based on my experiences you should use as long shutter speed (if it can be set on the camera) as the motor can follow the object without diversion and increase the value of ISO to a highest value (e.g.: 800).
The following pictures will be created by Samsung s760 at 50x magnification and using 8 seconds shutter speed with RA driving.
The first photo shows the Pleiades (M 45), the second one the Andromeda Galaxy. I think these are the best for this compact, if you want to take better photos, you need to get a DSLR camera.
Now, here are a few photos about the Moon created by Scopium camera. As you can see, the resolution was less thus the photos contain only a little part of the Moon but they are sharper and more detailed even at higher magnification. If you would like to see the full Moon, you need to use focus reductor.
The third photo was created at higher magnification, the diameter of crater on the middle of the picture is “only” 100 km (Theophilus crater), and the minor crater on the right side of this is 24 km (Madler crater):
Update: I’ve uploaded my sky photos to flickr: pzolee’s sky photos
[.] More
