Exception #0815 in function "subtitle_generator", Error: Visitor IQ too low!

Archive for the ‘Projekte’ Category

Spamassassin bayes folder/DB permissions for autolearning (and enabling the same when manually moving mails to Junk through a dovecot sieve)

Saturday, June 11th, 2022

This is another reminder to myself in case I ever have to re-setup the mail system. Fingers crossed that this doesn’t happen any time soon! But if it happens this post will hopefully spare me another few weeks of searching outdated discussion board and mailing list posts until finally reading the proper documentation… :/

I am running a Debian 11 VPS with postfix, dovecot and (daemonized) spamassassin. Only recently had I discovered that spamassassin’s bayes filtering and autolearn functionality was not working properly for multiple reasons:

  1. By default the bayes database lives in /root/.spamassassin which is … suboptimal.
  2. The bayes database needs to be trained with at least 200 spam+ham mails to be used at all .
  3. In my setup not only the system user debian-spamd would access the bayes DB, but also the dovecot/vmail user would attempt to access it when regular mail users would manually move a missed spam mail from their inbox to the Junk folder.

Fortunately, each step was easier to solve than I’d like to admit:

  1. Searching the web reveals that /root/.spamassassin is a very unfortunate place for the bayes database because most likely the daemonized spamassassin is not able to write there. This appears to be a common problem as I have found this issue quite often during my “research”.

    1.1 Move the bayes DB to somewhere else, e.g. /var/lib/spamassassin. The spamassassin option you are looking for is “bayes_path” in /etc/spamassassin/local.cf: https://spamassassin.apache.org/full/3.0.x/dist/doc/Mail_SpamAssassin_Conf.html

    1.2 There already exists a dedicated user to run spamd: debian-spamd, but for some reason it is not used by default. This can be fixed in /etc/default/spamassassin by appending “-u debian-spamd” to the OPTIONS line. After restarting spamassassin it should now run as the correct user.
  2. “Luckily” one of my mail users had a large number of spam mails in his Junk folder which were correctly classified and moved there by the default non-bayesian spamassassin rules, and could be used to train the >200 required spam mails for the bayes filter:
    https://spamassassin.apache.org/full/3.0.x/dist/doc/sa-learn.html
    The same was done for the ham mails.
  3. This was the tricky part: After moving the bayes_path and setting up spamassassin to run as the correct user, the /var/lib/spamassassin folder had these permissions:
    4,0K drwxrwxr-x 8 debian-spamd debian-spamd 4,0K Jun 11 12:57 spamassassin

    This was good enough to have mails recognized as “ham” by the now running bayes filter to be auto-learned as such. However there was another player in the field that would try to update the bayes database: dovecot/vmail.
    I had set up an IMAPSieve for dovecot which is supposed to automatically “sa-learn –spam/ham [mail]” when being moved manually from either Inbox to Junk or vice versa: https://doc.dovecot.org/configuration_manual/howto/antispam_with_sieve/
    While spamd itself is allowed write to /var/lib/spamassassin thanks to the updated permissions (1.1), the dovecot sieve could not. Turns out the sieve invoked on a manual move runs as user “vmail”, resulting in error messages in the dovecot log that the sieve execution has failed.
    Searching the web again showed many suboptimal solutions, including recursively setting the permissions for /var/lib/spamassassin to 777… That’s not what I want.
    Instead:
    3.1 Add user vmail to the mail group (if it isn’t already)
    3.2 Change ownership of /var/lib/spamassassin to debian-spamd:mail
    3.3 Change ownership of /var/lib/spamassassin/bayes_* to debian-spamd:mail
    3.4 Change permissions on /var/lib/spamassassin to 775/770
    3.5 Change permissions on /var/lib/spamassassin/bayes_* to 664/660
    3.6 Bonus: If you have enabled the spamassassin CRON job in /etc/default/spamassassin:
    Change permissions on /var/lib/spamassassin/sa-update-keys to 700
    to avoid warnings about unsafe permissions. Your /var/lib/spamassassin folder contents should now look like this:
    > ls -l /var/lib/
    […]
    4,0K drwxrwxr-x 8 debian-spamd mail 4,0K Jun 11 12:57 spamassassin
    […]
    > ls -l /var/lib/spamassassin
    […]
    356K -rw-rw-r-- 1 debian-spamd mail 360K Jun 11 12:57 bayes_db_seen
    7,6M -rw-rw-r-- 1 debian-spamd mail 10M Jun 11 12:57 bayes_db_toks
    4,0K drwx------ 3 debian-spamd debian-spamd 4,0K Jun 9 00:33 sa-update-keys

For me this allows spamd itself to learn ham from legitimate mails. It also allows the bayes DB to be updated when manually moving missed spam mails from Inbox to Junk using the dovecot sieve.



Date Format Conversion

Sunday, June 11th, 2017

Shortcut: Javascript Date Format Conversion

I recently had trouble to convert a date column from a CSV file to another (sorting-friendly) format. In LibreOffice >=5.3.x. In my case I wanted to convert from M-D-Y with no leading zeros, “/” as separator and 2-digits year to something like “yyyy-MM-dd” (i.e. 4-digits year, leading zeros and “-” as separator) for importing the file to another CSV file. I wanted to sort this document by date which doesn’t work with the M-D-Y format.

What I had: M/D/Y, no leading zeros, slash-separator, only two-digits year)

What I wanted: Y-M-D, leading zeros, – as separator, 4-digits year

 

What I got (nothing happened at all… )

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

I have tried numerous combinations of CSV import options and conversion parameters. I have spent some time googling for solutions, but most of them were dirty hacks and provided no permanent solution. Maybe it is related to the fact that my Ubuntu’s language is set to English(US) but the date/time/number formats are set to German. In the end I gave up.

And made this:

jdfc

Javascript Batch Date Format Conversion https://dfc.dennis2society.de

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Javascript Date Format Conversion



OpenCV’s TextureFlow Example in C++

Friday, March 29th, 2013

OpenCV's TextureFlow example screenshot (C++ variant)

OpenCV’s TextureFlow example screenshot (C++ variant)

 

Since version 2.4.3 OpenCV has a nice example for generating a texture flow image from an input image. This generates a grid-like distributed edge-gradient image from the input. Unfortunately this example is in Python with no C++ translation available. A while ago I have made an attempt to translate this to C++. The result is not 100% identical to the Python variant, but close enough:

 

 

 

 

 

Read the rest



QTetraMesher – A Tetrahedral Mesh Generator for Windows and Linux using Qt

Thursday, March 7th, 2013

This is a Qt-based program for Windows and Linux to generate tetrahedral meshes for finite element simulation from various surface mesh formats. It also offers a fast and easy-to-use mesh viewer based on QGLViewer and allows basic mesh manipulations (currently only scaling is possible). Two different methods for tetrahedralization are possible: Delaunay Triangulation and Johnathan Shewchuk’s Isosurface Stuffing algorithm.

This project has its own page: http://qtm.dennis2society.de/

qtm_with_CGAL_options_small qtm_with_stuffing_options_small



Painless Tetrahedral Barycentric Mapping in C++

Sunday, August 14th, 2011

Screenshot of a liver surface mesh mapped to a tetrahedral mesh

This article provides a set of functions to generate barycentric coordinates for surface mesh vertices in relation to a tetrahedral mesh. Additionally I provide some more functions to find the closest tetrahedron to a certain point P. As barycentric coordinates are invariant to translation, rotation and scaling, barycentric mapping is especially useful for efficiently updating a surface mesh according to an underlying deformable tetrahedral mesh.

This article was born out of frustration as I could not find any comprehensive guide on how to do this and I didn’t just want to “borrow” someone else’s code without really understanding what it actually does ( e.g. from SOFA Framework or PhysX SDK). It is primarily meant as an extended reminder for myself, but might be useful for other people dealing with tetrahedral meshes. This article will mostly contain a lot of code and only very few theoretical background and no mathematical proofs! At least that’s what I regard as “painless”. The advantage of my approach is that it simply relies on Vec3f and Vec4f types and avoids funny constructions like “MeshHash” as seen in the PhysX SDK. This will allow easy adaption to different VecXf type implementations and simplify integration into different applications.

For best understanding you should at least be familiar with the following concepts:
Read the rest



Android EZBridle

Sunday, August 14th, 2011

EZBridle Screenshot Interactive Mode

EZBridle Screenshot 01

EZBridle is a free Android application for calculating 2-point rigging suspensions aka. bridles.

Highlights:

  • Interactive Mode: Drag the beams and the A-pack to match your measures and steel lengths.
  • Manual Mode: Manually enter all relevant parameters (beam heights, distances and chain length). This might be useful for fine-tuning your parameters or for people with fat fingers…
  • Automatically calculate the resulting bridle’s angle (maximum allowed angle is ~130°).
  • Preserves maximum chain length, i.e. there will be no stinger(the yellow line) unless your chain-length is exceeded.
  • No questionable permissions required! This means it will work offline (no internet connection required) and won’t access GPS data, contacts or personal data.
  • No advertising!

Read the rest



[2. Update] Tetrahedral Mesh Generation (GMSH) for SOFA

Tuesday, March 22nd, 2011

Update:

There is now a better variant, for this reason (and to get rid of download-crawlers… ) the download has been disabled.

 

During the past few months I have spent lots of time doing experiments using the SOFA framework. In the course of my work I came to the point when I needed some more tetrahedral meshes than those that came included with SOFA. For this reason I started to do some reasearch on how to generate tetrahedral meshes from surface meshes (such as .obj). There are in fact lots of papers available on how to do this, but none of them provided a publically available executable. Furthermore, common programs for tetrahedral mesh generation either do not support the GMSH format (which is at least my own favorite due to good support in SOFA), or require manual work because of wrong order in the tetrahedra indices or additional information that is incompatible with the GMSH format. Anyway, none of those tools provided a solution for surface-to-tetramesh generation.

The solution for tetrahedral mesh generation from .obj files was found with PhysXViewer that is included in Nvidia’s PhysX SDK. It is available for free, but requires registration. The PhysXViewer allows you to import a surface mesh in .obj format and generate a tetrahedral mesh in different resolutions with a simple button and some sliders.

UPDATE: To generate solid meshes you have to activate the “Iso SIngle” mode. Otherwise there will be only an outer layer of tetrahedra.

The resulting tetrahedral mesh can be exported as an XML file in the NXUStream2 format. This format simply has some XML-nodes containing the vertices and the tetrahedra-indices . As the tetrahedra-indices are in correct order, it was pretty easy to write a PhysX_NXUStream2_to_GMSH_converter tool.

Physx NXUStream2 to GMSH converter usage

Download: Download disabled (see above)

The downloadable archive contains the sourcecode, the VisualStudio-2008 project, a compiled binary of the resulting program and some demo files (a simple sphere .obj file, the NXUStream2 tetrahedral mesh and the conversion result in GMSH format).

This tool is using the TinyXML library.



Musicracer

Thursday, May 14th, 2009

This project deserves a dedicated page but should be at least mentioned here.

Visit musicracer.de

Visit https://musicracer.de



J2ME: Mobile Sprichwort-Rekombinator

Sunday, March 2nd, 2008

Mobile Sprichwort-RekombinatorDer Sprichwort-Rekombinator fürs Handy!!!

Inspiriert von http://www.sprichwortrekombinator.de/ musste das ganze fürs Handy umgesetzt werden! Denn man kann ja nicht jedesmal ins Internet, wenn man mal was schlaues sagen will….

Beschreibung: Dieses J2ME-MIDLet rekombiniert deutsche Sprichwörter. Zur Zeit gibt es ca 15.000 mögliche (Re-)Kombinationen.

Highlights:

  • Der frühe Vogel höhlt den Stein.
  • Unkraut studiert nicht gern.
  • Politik ist aller Laster Anfang.

Read the rest



WP-Plugin: dennis2society Advanced Math Captcha

Saturday, March 1st, 2008

Captcha-Sample

Description: WordPress-Plugin to prevent comment spam

In contradiction to conventional CAPTCHA systems this solution does not simply require to recognize distorted numbers or letters or maybe solve an addition or subtraction, but instead challenges the user ( or spambot ) to solve REAL mathematical equations such as derivation or a limit ( limes ) .

Captcha-Sample 2This is admittedly not totally barrier-free but will at least handicap the common spam bot. And people who don’t know about mathematics…
Read the rest