insomniac2k2 wrote:The google ip address that is being added is this. Notice the redundant entries that happen within milliseconds of each other.:
﻿6060 "2017-10-29 10:39:15.920" "Message from: 74.125.82.50 mail-wm0-f50.google.com Added as to Greylist Whitelist do to match in approved list"
﻿6560 "2017-10-29 10:39:16.639" "Message from: 74.125.82.50 mail-wm0-f50.google.com Added as to Greylist Whitelist do to match in approved list"

This is probably caused by you offering StartTLS on port 25.
other mail servers connect once, get offered StartTLS, and then connect securely, before connecting again.

Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Thanks SorenR. I'm aware of why you implemented that workaround. I just wanted to be sure why such a workaround would be necessary. As this is something that should be rectified in the server build. Matt's post answered my question. I did not think about StartTLS. That makes perfect sense. If i get some time, I will see what needs to be patched on the server to rectify this bug.

SorenR wrote:

SorenR wrote:If you look back in your other thread you'll see in my GreyWhiteList code i am using my eventlocking to avoid violating the database constraint.

insomniac2k2 wrote:The google ip address that is being added is this. Notice the redundant entries that happen within milliseconds of each other.:
﻿6060 "2017-10-29 10:39:15.920" "Message from: 74.125.82.50 mail-wm0-f50.google.com Added as to Greylist Whitelist do to match in approved list"
﻿6560 "2017-10-29 10:39:16.639" "Message from: 74.125.82.50 mail-wm0-f50.google.com Added as to Greylist Whitelist do to match in approved list"

This is probably caused by you offering StartTLS on port 25.
other mail servers connect once, get offered StartTLS, and then connect securely, before connecting again.

For now, until I get a chance to take a look at the server code, I just implemented some easy line parsing logic my application which will check the second to last line to see if a redundant entry is already in the Event log. If entry exists, then exit without trying to create the GreyWhitelist. Similar hack. It just offloads it from the eventhandler.

insomniac2k2 wrote:Thanks SorenR. I'm aware of why you implemented that workaround. I just wanted to be sure why such a workaround would be necessary. As this is something that should be rectified in the server build. Matt's post answered my question. I did not think about StartTLS. That makes perfect sense. If i get some time, I will see what needs to be patched on the server to rectify this bug.

SorenR wrote:

SorenR wrote:If you look back in your other thread you'll see in my GreyWhiteList code i am using my eventlocking to avoid violating the database constraint.

I'm only counting the log lines from the event log. It should rarely see traffic. And the traffic that it does see is only the events that I am writing to it. So far, that is only ban and greywhitelist information. That and at the same moment the GreyWhiteList is called, the read will happen. The probability for issue is rare atm. It can easily be made more robust if needed.

mattg wrote:to be honest only counting back a couple of log lines may not catch the first HELO/EHLO on a busy server, or even just a server doing something else right now

Yep, I find the logging useful. Many different ways to skin a cat. Personally, i think the real fix is in the server code. Without looking at it, i would hope that a boolean could be set if startTLS is initiated, to skip onHELO. Then allow OnHELO after the startTLS connection was initiated. This would also solve any other issues with script and update redundancies.

We shouldn't be writing workaround code at all for this type of stuff IMO.

insomniac2k2 wrote:Yep, I find the logging useful. Many different ways to skin a cat. Personally, i think the real fix is in the server code. Without looking at it, i would hope that a boolean could be set if startTLS is initiated, to skip onHELO. Then allow OnHELO after the startTLS connection was initiated. This would also solve any other issues with script and update redundancies.

What if sender do not support STARTTLS ?

SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

I suppose the easier way would be to process OnHELO on first connect and omit running OnHELO in startTLS.

SorenR wrote:

insomniac2k2 wrote:Yep, I find the logging useful. Many different ways to skin a cat. Personally, i think the real fix is in the server code. Without looking at it, i would hope that a boolean could be set if startTLS is initiated, to skip onHELO. Then allow OnHELO after the startTLS connection was initiated. This would also solve any other issues with script and update redundancies.

insomniac2k2 wrote:Then the bool wouldn't be set, and OnHELO sub would process.

I suppose the easier way would be to process OnHELO on first connect and omit running OnHELO in startTLS.

SorenR wrote:

insomniac2k2 wrote:Yep, I find the logging useful. Many different ways to skin a cat. Personally, i think the real fix is in the server code. Without looking at it, i would hope that a boolean could be set if startTLS is initiated, to skip onHELO. Then allow OnHELO after the startTLS connection was initiated. This would also solve any other issues with script and update redundancies.

What if sender do not support STARTTLS ?

Port 25 is used for server - server kommunication and I believe it is generally a bad idea to inforce TLS on this port until the rest of the world support it 100%. I don't use TLS at all and I can send/receive mal OK.

GreyWhitelisting is not for clients so by checking oClient.Port = 25 you bypass the code for your port 587 TLS and 465 SSL connections.

SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

Now that's the best one yet. I did not think about doing a port check at all! Ill throw that check in and omit the workaround to test. This would be the most efficient use of resources.

EDIT: I believe i spoke too soon. I believe that StartTLS just escalates SSL communication on port 25. Thus resetting the connection and causing an additional OnHELO. Brain fart moment!

SorenR wrote:

insomniac2k2 wrote:Then the bool wouldn't be set, and OnHELO sub would process.

I suppose the easier way would be to process OnHELO on first connect and omit running OnHELO in startTLS.

SorenR wrote:
What if sender do not support STARTTLS ?

Port 25 is used for server - server kommunication and I believe it is generally a bad idea to inforce TLS on this port until the rest of the world support it 100%. I don't use TLS at all and I can send/receive mal OK.

GreyWhitelisting is not for clients so by checking oClient.Port = 25 you bypass the code for your port 587 TLS and 465 SSL connections.

For my purposes, I also added logging to come from ProxyAuth instead of Hmail Event logging. Mainly because I do not have control on whether or not a redundant record gets logged (If something matches the criteria, it logs regardless of whether or not an IP address is being added to the GreyWhitelist.)

What I'd like to see (RVHD and SorenR) is there be some sort of information detailing the SSL statuses of a connection being made available via the API

Something like
(and possibly more widely available than just in this OnHELO sub)
Properties
oClient.encrypted (True|False)
oClient.SSLTLSVersion (SSLv3.0|TLSv1.0|TLSv1.1|TLSv1.2)
oClient.cipherused (cipher)
oClient.certificateName (name)
oClient.certValidated (True|False) - was the cert validated by hMailserver

Method
oClient.ForceCertValidation

Currently this is only available via the logs, which makes it impossible to use in real time. If I had API access I could script some more things
I'd like to do things like spam score based on these properties
Force certificate validation if spam score between 10 and 20, or even just above say 10
Only allow a single known IP/User to use SSLv3.0
not allow downgrading of TLS version during a connection
Only allow Outlook.com connections via TLSv1.2
etc

Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

I like those options, but they are likely a lot of work for a little payoff (guessing about implied code changes).

I'm curious about why after you spam score something so high, why would you need to verify their cert? Why not just reject them?

mattg wrote:What I'd like to see (RVHD and SorenR) is there be some sort of information detailing the SSL statuses of a connection being made available via the API

Something like
(and possibly more widely available than just in this OnHELO sub)
Properties
oClient.encrypted (True|False)
oClient.SSLTLSVersion (SSLv3.0|TLSv1.0|TLSv1.1|TLSv1.2)
oClient.cipherused (cipher)
oClient.certificateName (name)
oClient.certValidated (True|False) - was the cert validated by hMailserver

Method
oClient.ForceCertValidation

Currently this is only available via the logs, which makes it impossible to use in real time. If I had API access I could script some more things
I'd like to do things like spam score based on these properties
Force certificate validation if spam score between 10 and 20, or even just above say 10
Only allow a single known IP/User to use SSLv3.0
not allow downgrading of TLS version during a connection
Only allow Outlook.com connections via TLSv1.2
etc

My spam scoring system is extremely customised, one of my SpamAssassin rules adds 2.2 for no reason other than to make the scores closer to what matches my hmailserver scores.
Another SpamAssassin rule adds 30 for ClamAV + SaneSecurity fails (This is typically malware attachments)

I normally reject mail above 15 (via onAcceptMessage script on my system), and I am currently ALSO adding an Autoban for scores above 25
After a a couple of weeks with these high SPAM scores getting an autoban for a week - I have some ~370 addresses autobanned due to high SPAM Score.

Next time they connect, they just get simply blocked.

Most SPAM to my system arrives from StartTLS connections with valid DKIM and SPF records. I'd like to **selectively** validate the cert used for StartTLS too, and then spam score against the result.
They may all use valid SSL certs, but at the moment, I've really got no way to tell. I also have one machine (a security camera) that will ONLY use SSLv3.0. This camera is on a static IP address and I'd like to allow that, but ban all other IPs from using SSLv3.0 as it is compromised.

The only way presently that I can see if a connection was StartTLS or not is to check my logs after the connection. Bit late then to do anything with the information.

Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

I have actually blocked SSLv3.0 as the camera mentioned above no longer sends via my server
Only 3 IPs have connected via TLSv1.1 so far this month. One of these I have received 8 messages from - so a regular sender (and yes I checked which sender)
I have quite a few that connect via TLSv1.0, including Facebook, and my youngest daughter who has my old (now forth hand) iPhone 4s. Newer iPhones seem to use TLSv1.2

As an aside, I force TLSv1.2 on all of my Apache hosted websites, and found a stock Internet Explorer ver11 on a Windows Embedded HP thin client the other day that was capable of using TLSv1.2, but had TLSv1.1 and TLSv1.2 disabled, but allowed SSLv2.0 and SSLv3.0, and so connection failed. I'm trying to find a way to show different pages based on SSL/TLS level used, or even a default connection failed page - but this is proving to be hard.

Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

If so, you have my exact same configuration. I score virus as 50 and malware of 10. Anything over 10 i am presently discarding. In the future I will be blocking the sender, etc. I'm quite happy with the way im configured. I cannot recall the last spam message I have received

I like the thought of discarding more connections. I certainly would reduce the load on our servers.

mattg wrote:My spam scoring system is extremely customised, one of my SpamAssassin rules adds 2.2 for no reason other than to make the scores closer to what matches my hmailserver scores.
Another SpamAssassin rule adds 30 for ClamAV + SaneSecurity fails (This is typically malware attachments)

I normally reject mail above 15 (via onAcceptMessage script on my system), and I am currently ALSO adding an Autoban for scores above 25
After a a couple of weeks with these high SPAM scores getting an autoban for a week - I have some ~370 addresses autobanned due to high SPAM Score.

Next time they connect, they just get simply blocked.

Most SPAM to my system arrives from StartTLS connections with valid DKIM and SPF records. I'd like to **selectively** validate the cert used for StartTLS too, and then spam score against the result.
They may all use valid SSL certs, but at the moment, I've really got no way to tell. I also have one machine (a security camera) that will ONLY use SSLv3.0. This camera is on a static IP address and I'd like to allow that, but ban all other IPs from using SSLv3.0 as it is compromised.

The only way presently that I can see if a connection was StartTLS or not is to check my logs after the connection. Bit late then to do anything with the information.

mattg wrote:After a couple of weeks with these high SPAM scores getting an autoban for a week - I have some ~370 addresses autobanned due to high SPAM Score.

Now Autobanning at 22 or up (as opposed to 25 when posted above), with no upper limit (I was rejecting outright at 100 and above, have now had 2 mail messages score more than 100 points), and and I have some ~210 addresses autobanned due to high score

From 370 down to 210
I'm getting significantly less spam senders

so why?

Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Most spam is sent from infected spambots and those are mostly on peoples work PC's. This is obvious by the fact that the levels drop significantly (to almost non-existent) during public holidays (such as Christmas etc) only to resume in anger on Monday mornings or the first working day back. Also I have noticed that they go in fits-and-starts which I believe to being the ISP/hosting networks uncovering and shutting down the suych bots as well as the AV solutions followingand eventually catching up with their definitions.

This I conclude after watching over recent years. And you can be sure that your lower numbers will, one day, have a surge again when some dick out there releases some new virus/spambot going undetected for a while before getting shutdown some months after.

Since installed this latest built the other day, My server has commenced 'shutdown' at 5:40 am, and stopped accepting new connections. About a half minute after doing a POP3 external download.
I'll go back one version.

(No other changes - Thanks RvHD for the builds)

Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

I (still) can hardly believe this has something to do with these changes as i never experienced something like this, and yes i use external download as well
You appended the file locking thingy to your scripts?

I have a scheduled task that runs daily and backups up my hMailserver, zipping messages by domain, and dumping MySQL tables etc.
I've been doing that for years without much change - it works.

I've recently switched to Lets Encrypt certificates, that are created automatically by my web server.
In hMailserver I link to a folder on the web server where these are stored. Works great.

To be sure that the latest cert is always loaded, at the end of my scheduled task for backup, I added an extra action of running this script. The scheduled task doesn't complete. The scheduled task is run as my usual desktop user, and the script ALWAYS runs fine when manually run. Any ideas about why it sometimes fails when run as a scheduled task 'action'.

This is intended to pause and re-start the hmailserver, as opposed to restarting the service which I don't want to do as I leave the Admin GUI open.

I think it is your Wait() function causing issues, i have been using that pure vbscript approach myself in the past and it gave me nothing but trouble (examples: running longer as defined, or even running infinitely)

Yes I did change to that version of Wait, and hadn't had an issue until about 48 hours ago when I've had a single failure.
I'm working on finding out when it happens by checking the last 15 lines of the log and if they all say 'No message to index" then I'll reboot the hmailserver service . Not quite got it working yet but close.

Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Just updated to the latest beta official update (hMailServer 5.6.7 - Build 2425 (BETA)) and was going to modify it with latest RavdH implementation of code when I saw mattg message here and was wondering if there is a problem with the latest unofficial update. Should I go with one version before this?

I would say there is no problem with these builds as he is very careful to implement changes that have been passed and proven by the author in upcoming beta and 5.7 release. Matt's problem is isolated and is unclear on what the cause is but is likely to be specific to his setup/scripts (he is monitoring and investigating). Others are also using both Sorens suggested script and this ed executable offered by rvhd without a problem. You could save time and simply install this 5.6.7-B2425.15 version instead of modifying your own (from beta) to benefit from all the fixes and mods.

Thank you, downloading RavdH latest version right away . Autoban from SorenR is simply brilliant, will search for examples of OnHELLO sub to call autoban, saw a couple of topics here with different implementations, I can only say thanks for all of you that put hard work into making and maintaining this.

Once you have the basics running you'll figure it out by monitoring your logfiles. Every environment is different i.e. it is unlikely we share the same bots/spammers so... dig into your logs and go from there.

SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

I've just added a domain and host two accounts for that domain, and the rest are at the ISP of the client.
Normally I'd just set up a route for the domain and select when sender matches route treat as 'Remote' and this would allow someone else from that domain send me mail at my domain also hosted on my hMailserver.

Today this isn't working...
They use Outlook2010 as mail client

I tried the ini setting 'AuthUserIsLocal=' as both 1 and 0 without change

The only way that I could get it to work was to allow local to local without AUTH on the appropriate IP range.
Could this have something to do with the route changes that you have made?

Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

RvdH wrote:issue #74 maybe? Don't (yet) use this myself...
Do you have a catch-all account locally for that domain?

No catchall for that domain.

I can roll back to the official build to see if the issue exists there. I normally don't use this functionality, but found a need this week.
I did find a recent user post stating the same issue with the official builds - so perhaps it is a real bug.viewtopic.php?f=7&t=32256&p=201556#p201556

Does anyone routinely use routes for partially hosted domains?

Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Thanks for the work on this, I really like the OnClientLogon event for logging login attempts. Any chance to allow a return result like onClientConnect? I have some cases where I do not want a specific account to be accessed by a specific IP, but allow that IP to access other accounts: https://www.hmailserver.com/documentati ... entconnect

Having the option to specify Result.Value = 1 would solve that for my scenario.

Indeed, its the chicken & egg, OnClientConnect doesn't yet know the username and OnClientLogon doesn't have the return Result feature. I would like to block further activity for a specific user from a specific IP but I don't want to block all users from that IP address.

Indeed, its the chicken & egg, OnClientConnect doesn't yet know the username and OnClientLogon doesn't have the return Result feature. I would like to block further activity for a specific user from a specific IP but I don't want to block all users from that IP address.

thanks

I will try and see what i can do with return values whenever i have some spare time...i think i left it out on purpose as this needs to be handled in SMTP, IMAP and POP protocols

Thanks, I understand the complexities of bringing in the protocol to close the connection cleanly. Disconnect.exe could conflict with another user if its the same IP address, but that might be OK in this context, I'll give it a try.