When someone wants to create a custom Joomla extension, one of the options is to use one of the several online tools that allow users to input the particulars of their concept and have the tool do a lot of the heavy lifting in terms of directory and file generation. A Joomla extension is a thing of many parts. They share common directory structures and files with similar code. Some clever clogs have decided that the repetition of tasks involved in their creation is too much to bear, and have automated the process through these online tools. The great thing is that they've made these tools available to everyone, and they are an enormous time saver for those involved in creating custom extensions for the Joomla CMS.

In recent months, I've become increasingly obsessed with whois spam. It's become a bit of a moral crusade for me. I don't really understand why the cost of integrity and transparency in the domain registration system has to mean that my inbox is filled with spam, and I get dozens of calls daily whenever i register a new domain. Adding to my frustration is the conventional answer- a domain privacy add-on, effectively doubling the cost of my domain registration. This rort is purchasable on an annual basis, and on a per-domain, not per domain-owner, basis.

Pay for your new home, then pay someone else some money to make sure your mailbox isn't vandalised.

So, never being the person who goes for the obvious answer, I've decided to conduct a little experiment. I'll try to create a system to shield my phone number and email account from endless offers for SEO services and web designers.

I've set a a dedicated domain (domainbead.com) for the project, and an email address which I've used on the whois register for a number of my domains. I'm devising a number of filtering mechanisms that you can read about here.

I've also arranged a VOIP virtual receptionist and voicemail system to handle phone calls, with a few mechanisms to weed out the spam callers.

If you've ever been annoyed with whois spam, and you'd like to follow along, then drop by https://domainbeard.com from time to time over the next couple of months. If you're really keen, or if you think you have something to offer to the project, then you can get in touch and we can work on tackling whois spam in a new way together!

If you've been wondering why a world of web site developers, CMS teams, shopping cart providers, and blue chips are all maintaining their own independent geographical databases, then you're not alone.

Imagine if every website wanting to display the weather forecasts, forex information, or sporting results all had to download CSV files from a github repository and then commit themselves to maintaining their own lists forever more.

Countries, states and cities may not pop into existence as often as the weather changes, but the idea that every website should maintain their own database seems crazy. It's the kind of thing that is ideally suited to a web service.

Updating country lists may be a mere irritation, an occasional chore, for a website administrator. After all, the last time a new country needed to be added was back in 2011, when South Sudan gained independence from Sudan. However, states and their names change much more frequently, and towns and cities even more so. And then there's additional information about locations, such as their population, that change even more frequently.

The point of Geodata Solutions is to make this tedious task a job for one organisation, rather than for millions around the world. I plan to build on the existing database I've created, keep it current, and build on its assets. My hope is that this project will change the way geographical information is sourced in web applications. Syndicated usage of the system will alleviate the duplicated burden of maintaining local data stores, while allowing error reporting and updates to be submitted by millions of users.

The RESTful interface is in 'Beta' at the moment. It offers country lists, state lists and city lists, along with some basic additional location information. Developers wanting to use the system should get in touch for an API key, as well as requests for new search types or filtering, or new pieces of data. There's also quite a big job in completing gaps in the data set, of which population data is a significant one.

The calls are starting to die down now, and I only get 5 emails a day now, rather than 20. That's some comfort, I guess.

I've had to ensure the same process that most proud new domain owners endure. Calls from allegedly local numbers from offshore web developers, SEO services, social media marketers and app builders offering their services towards making my new domain the successful new venture that they assume i want it to be. Emails too. Many more emails than phone calls. In the week following the registration of three domains- one '.com' , one '.co.uk' and one '.xyz' - I was receiving around 20 emails a day, but a mere 5 or 6 daily calls.

Much has already been written on the issue of whois spam, and whether the balance between the integrity of the web and privacy. I would suggest that having a name and postal address would be sufficient in order to bring accountability to domain ownership and ensure that those with valid reasons for contacting you are able to do so. This adds a small cost to the contact process and avoid email spam and nuisance calls.

Domain registrars acknowledge this issue with the whois registration, and use it to promote their various domain privacy options, which often cost more than the domain itself. No doubt, on a day with 7 nuisance calls and having my inbox filled with dozens of spam emails, I would have told you that the price of these domain privacy packages was absolutely worth it. But then again, why should we have to pay a significant levy on the price of registration to avoid a form of harassment that is uninvited and entirely caused by an ill-considered public register policy.

So I wondered what I could do about it. How could I devise my own system that detected and filtered out spam and nuisance calls, while still providing full disclosure of ownership and a means to contact the owner with legitimate queries. So I wrote down patterns and wondered how i might address them. If I was to create a phone number and email address that served an as intermediary to my usual number and email address, what are the means by which I might be able to ensure that I might be able to filter out dubious connections and allow through correspondence I need to know about. Because the email address and phone number would only be used for whois publication, I might be able to apply customised filters that would be impractical to everyday email or phone call screening processes

Pattern

Problem

Solution

Calls often received without caller ID

This may suggest that the user is calling from a VOIP service or is actively obscuring their number. Although there may be legitimate reasons for doing this, it usually suggests that the caller has a reason for not wanting to reveal their identity

The phone number listed on the whois register would take the caller to an Interactive Voice Response system that would advise them that calls without caller ID are not permitted and that they should call back from a disclosed number

Emails often come from public email provider addresses

It's not without irony that people purporting to be web professionals send you their sales pitch from a gmail or hotmail account.

Along with the fact that these emails rarely contain the name of the company or links to their portfolio, it indicates that the sender is aware that they are engaging in spam.

The email system would add a greater level of scrutiny to email messages coming from these addresses. Senders would be sent a captcha image which they would need to respond to and confirm by email in order for their message to be received. This added burden makes blanket emails impractical.

Emails contain similar keywords

Emails offering web services contain the same words and phrases.

A database can be easily created with keywords that indicate a sales email and this can be run against all incoming email to filter it appropriately

So, I've created domainbeard.com, where I'm staging my fightback against whois spam. I have a IVR call system set up that leads all legitimate callers to a voicemail which alerts me to new messages by email, and i can pick up at any stage. I also have a dedicated email address which I've used in the registration of a few new domains. I'm monitoring and configuring an email system that necessitates validation and emails e senders a response requiring their action under certain circumstances. I'm just learning now, but I see a place for a new free service for all domain registrars to use that will alleviate the pain of whois spam.

Allowing legitimate correspondence

I figured I could not only create a system that filters out junk, but also identifies high priority calls and emails

Pattern

Solution

Calls from bonafide sources come from recognised numbers

A database of numbers can be built up of both known nuisance callers, and conversely of known domain regulators and legal bodies. This could rely on the reporting of users and be built up over time

Emails from bonafide sources come from recognised email addresses

The same applies to email. A database of known spammers and known legitimate sources would be built up which would enable better filtering as time goes on.

At the moment, I'm gathering data and experimenting with different approaches, but I'll be opening up Domainbeard to any domain registrant early in 2018 with a few main goals:

Alleviate the pain of spam email and nuisance calls by providing an intermediary address and number specially configured to identify the legitimacy of sources

Allow users to report on the legitimacy of callers and email senders to improve the filtering algorithm

This can be used to save future email registrants from being bothered by known spammers

It can also be used to build a case of evidence to provide to ISP and telephony providers in order to have the account of spammers stopped.

Provide statistics and advice related to whois spam that is far more authoritative and quantitative than the anecdotes of a frustrated domain owner.

To be free- given the enormous wealth of data that could be collected, its ability to improve the integrity of the web, and a common acceptance that people ought not be subjected to nuisance calls and emails, why should people have to pay for this?

I'm pleased to announce that version 1.0.1 of the Geodata Solutions Location List Custom Fields Plugin is now completed.

Joomla site owners can now add location data to their content, categories, user and contacts components.

There were some difficulties along the way. Given the newness of the custom fields functionality, there is little documentation for the new plugin type, and the routine for adding new custom field types differs from the custom fields types offered in the default Joomla package, which have their templates hard-baked into the core. I have to thank people on the Joomla forum, who were able to assist with questions and offer support.

This is just the first iteration of the extension, and there are many improvements to come, within the functionality of the plugin's code, and also in the web service offered on the Geodata Solutions side.

A few 'hacks' were required in this version:

The jQuery chosen library had to be subverted in order to get the ajax functionality of the lists (as coded in general-use code snippet version) to work within Joomla's back end. The ability to 'destroy' the -chzn elements was available in Joomla 2.5, but it seems this was removed in J3.x. This version of the extension uses a hack that hides the chosen elements and unhides the original form fields offered by cavarief here.

The system uses the simple javascript request method that the general use code snippet uses. The disadvantage of this is that it doesn't allow for secure serverside authentication. At current usage, there is no need to throttle access to the web service, but I anticipate that when usage increases, I'll need to switch to a superior solution. Some things to note:

Authentication currently checks the HOST header in the js request- it's the crudest way of recording the usage of each separate implementation of the system, but hardly the most reliable

Proposed solution in the next release would be to have a key generated with each download of the plugin (I think I can do this without requiring registration with Geodata Solutions- something that I think is a barrier to quick implementation) which the user could simply enter into the plugin configuration

Very high volume sites might have to subscribe to support the service at a later stage

There are issues related to the output/override of custom fields in the user component. This is reported issue, and one that I hope will be corrected in future Joomla version. Until then users will have to put up with the pipe-separated format used for stage within their user profile views in the front end (eg. Australia|Victoria|Ballarat ) or create their own template override. If you want some help implementing this, or want to share your solution, then please get in touch.