FYI, with regards to updating wtfs. it might be better to have a wtf_adjusted column in the weights log, therefore you have a record of both the automatic and manual wtfs. Inevitably you have to be careful when re running phases, as if you made adjustments to the weights log then reran phase1, it would overwrite, this is why you should always use --interactive mode and carefully read the messages as to whether particular files are empty or not. Alternatively adjust the filenames in the configuration so that the inputs and outputs named differently and you'll have to do some copying between phases.

FYI, with regards to updating wtfs. it might be better to have a wtf_adjusted column in the weights log, therefore you have a record of both the automatic and manual wtfs. Inevitably you have to be careful when re running phases, as if you made adjustments to the weights log then reran phase1, it would overwrite, this is why you should always use --interactive mode and carefully read the messages as to whether particular files are empty or not. Alternatively adjust the filenames in the configuration so that the inputs and outputs named differently and you'll have to do some copying between phases.

Chris

Hmmm... more to chew on.

FYI, I kept wondering about the number '27' that showed up in one QSO entry in the NAME column. Went back to the original log that was emailed (and subsequently 'fixed' and re-fixed' to remove unwanted QSO numbers). In my rushed delete/copy/paste work, I found the problem. Just re-uploaded another corrected entries.txt file:

http://www.xgenesis.com/hashorama/entries.txt

To get a better perspective of things and run more tests, the one thing that definitely would help right now is to at least have a NOLOG Column added to the Error log - to the right of the WTF Column, so that *any* Error where the CALLWKD callsign is a Non-Log Submitter is flagged NOLOG. This will also help me to refine the anticipated manual decision/processing to be done here!

I will re-visit the unsubmitted log stuff later after running more tests. I can just setup a new sub-directory here and use the new entries.txt file, and substitute a new main.pl file.

To get a better perspective of things and run more tests, the one thing that definitely would help right now is to at least have a NOLOG Column added to the Error log - to the right of the WTF Column, so that *any* Error where the CALLWKD callsign is a Non-Log Submitter is flagged NOLOG. This will also help me to refine the anticipated manual decision/processing to be done here!

Due to the issues and weights involving Busted calls/Non-Submitted Logs, here is what makes MUCH MORE SENSE and will be much cleaner:

Code

Instead of NOLOGS, the column should be LOGS and if the CALLWKD Callsign on any line of the Error Log is for any LOGCALL (i.e., a log was submitted), then what gets entered in that column is "LOG".

This will be a quick visual aid in my MERP (Manual Error Research Process).

This will be a quick visual aid in my MERP (Manual Error Research Process).

By having the new LOT column 'flagged', I can group my manual research tasks.

1. If a (submitted) LOG related Error, I will do those as a group and can quickly go through the actual submitted logs based on the sorting order (which was a reason for it :^)

2. If NOT flagged as a (submitted LOG related Error), I will do those as another group. For this group which includes possible email verification steps, I will use:

A. The CNQ weights.txt list which shows multiple CNQ combination errors for the same Callsign

B. An online HAM specific database with Callsign (owner & QTH) info. When a nickname is used, only the QTH info would be valid but a help.

C. For all USA callsigns (the majority in the logs), there is also the FCC Universal Licensing System online (Government) database.

99.9% of the time, if a callsign does NOT show in #B above, it's a plain old bad/busted callsign. Example: As of this posting, when I type in N4AFY, here is what happens:

Code

The search for "N4AFY" produced no results.

"BINGO!!!"

Likely the person's fingers slipped on the keys and added a 'Y' to N4AF's callsign. Of course, there are CNQ combo multiple entries in the weights.txt file for N4AF which is another issue to deal with {SIGH}.

I hope this helps better understand how I plan to use the Error log details schema in my 'MERP'.

- Added LOG column to errors log but I thought I would make it a bit more intuative. Instead of just outputting LOG where the call is valid and has an associated submitted log I have gone with 3 statuses. Let me know if this doesn't satisfy you. ----- '1' = callsign is valid and its associated log was submitted. ----- '0' = callsign is valid but its associated log was not submitted. ----- '-' = callsign is invalid therefore no log available.

I had it with the tax work for today/tonight, so just installed and took a 'Test Drive' of the latest version.

In Reply To

----- '1' = callsign is valid and its associated log was submitted. ----- '0' = callsign is valid but its associated log was not submitted. ----- '-' = callsign is invalid therefore no log available.

Interesting possibility - will let you know after more use.

In Reply To

- Included names and qths in unsubmitted log as this may be helpful.

Using K2QBN as an example, it shows 'EVAN' as the name, but I remember there was also a CNQ with 'VAN' in the Weights file. Not sure yet how this will be used compared to the Error log which I believe is the Mega Important tool...along with the weights.txt file list which shows me all the CNQ combinations. Yes, the latter is also Mega Important in my tedious MERP tasks.

But let me tell you what is really E-X-C-I-T-I-N-G ... and I almost soiled myself when I realized how COOL (and a productive time-saver). Can you guess?

By putting the QID numbers next to each line in the Error log, I no longer have to go searching through multiple log entries (even in band & time sequence) by LOGCALL. I can now go DIRECTLY to the specific Error log related QSO line I need to scrutinize. Talk about EFFICIENCY !!! Chris, this is BEYOND AWESOME !!!

I don't even have to type the Error log QID... just copy it and with the entries.txt file open, quickly do a CTRL + F followed by a CTRL + V and then pop the ENTER button. "BINGO!!! ... almost like Magic I am whisked right to what I need.

I can't tell you for years how many extra hours were spent flipping through an unwieldly stack of printed-out log sheets in the Nightmare log-checking process. Even later using individual .txt files on the Confuzzzer was a real pain having to close and open separate files. This is going to save a LOT of time. Dunno why I didn't think to consolidate all the logs into one 'Master' log way back when (DUH!).

Thanks sooooooo much for adding the QID's to both the Error Log and the Phase 0 assignment of QID's to each of the QSO lines in the entries.txt file !!!

I have actually changed the LOG codes to suitable labels instead: - 1 is now submitted - 0 is now unsubmitted - - is now invalid

Quote

Using K2QBN as an example, it shows 'EVAN' as the name, but I remember there was also a CNQ with 'VAN' in the Weights file.

Remember at phase2, after the weights log was generated at phase1, only names and qths that are >= wtf threshold will be included ( the same information as the scores log ). VAN has a wtf of just 1.

phase2 being the core automation phase ignores anything it deciphered to be invalid, afterall thats automation for you. You should always keep this in mind, the logs generated by phase2 are automated outcomes, don't expect anything the algorithm deciphered to be "useless" to be included.

The log that should be most useful to you is therefore the weights log, since this allows you to fine tune the outcome of phase2. Perhaps we do need to consider extending info in the weights log and / or breaking it up into groups and / or generating other consolidation logs at phase1, enabling you to make the right decisions before phase2. We have already discussed this in some detail, but I don't think we really finalised any ideas.

Also the QID's do indeed make lookups super quick, I used to look at an error call sign, then forget it by the time I opened up the entries, or atleast confuse it with one similar.

One other thing I was thinking was in the scores log under the scores column, it might be nice to do "score/max score", where max score would be the score they would have gotten if they made no errors whatsoever, then perhaps a percentage column i.e. ( ( 100 / max_score ) * score ). I'm not sure if this would be useful to you, but a lower percentage would be a good indication of who is the most error prone, while those with 100% percentage deserve a reward, perhaps a billion bonus points ;-). But of course those with a score of 0/0 would get 100%, and those who didn't submit many entries might be notably less error prone.

Also the QID's do indeed make lookups super quick, I used to look at an error call sign, then forget it by the time I opened up the entries, or atleast confuse it with one similar.

Yes, 'been there, done that - still doing that', but in my excitement, it was not until I took a needed R 'n R break that I realized the actual QSO line from the *other* (CALLWKD) station is also what needs checking. And at this point of 'Beta Testing', having the ability to do a rapid copy & paste into the FIND dialog box on the entries.txt page to zip right to the CALLWKD station QSO details would be great. Especially, for investigating the 'NIL' Errors.

So another idea came to mind: QID2 (or 'Son of QID' :^) If we added one more CWID (Call Worked ID) column to the far right of the Error log, this could accomplish the mission like with the CID. Of course, a CWID number would only display IF the Error line involved a CALLWKD by another log submitter. Yes, this would be Awesome(2) and save even more time.

FYI, when the logs are originally submitted, there is a "Claimed Score". If you look at that PHP Form URL again I PM'd you, I had already planned to use the this data as part of (manually) doing more tests... to make sure nothing fell through the cracks. Of course, the submitter's math skills (or lack thereof) have sometimes not been accurate. Not to be 'the pot calling the kettle black' since I've made more than my share lately by rushing too quickly.

I had actually considered including the 'Claimed Score' as another column in the categories.txt file. Ohhhh... now I'm getting another idea for something very useful, but will chew on it a bit more.

Based partly on the K2QBN example of CNQ weights of 3 and 1, I'm pretty certain already that a WTF of 3 would be the minimum. I also want to run tests at 4 and 5. What is also coming out of this is likely to be a tweak to next year's Rules.

The VE3|ON, VE4|MB and XE|DX situation is something I am still mulling over in consideration of the objectives of helping others also increase their on-air *and* logging accuracy vs. blanketly Auto-Mulliganizing any QSOs. This is a tricky one.

I have actually changed the LOG codes to suitable labels instead: - 1 is now submitted - 0 is now unsubmitted - - is now invalid

This is still baffling me a bit. I understand 1 and 0, but for example:

Quote

427 K6NV VE3KI 80M 0242 RICH ON CNQ 1<3 -

VE3KI did not submit a log, but a determination cannot be made as to whether the QSO is actually 'invalid' or not until investigated.

Checking the primary online HAM database, VE3KI is 'Richard' in ON(tario). Most likely he uses 'Rich' on-the air. This type of QSO may have to be treated as a 'Unique' vs. an invalid CNQ, and I'm continuing to chew on these situations in terms of disposition.

Hmmm... it might be more helpful to only flag the '1' lines as LOG (much clearer than trying to remember pseudo-codes).

Looking at this one flagged '0' ...

Quote

385 K6DGW W1NN 40M 0213 HAL SC CNQ 1<3 0

The MERP will be the same as for those currently flagged '1' as compared with a rapid QID/CWID review of actual submitted QSO lines and both these could be left blank. The only entries which would show in the CWID column would be LOG ('1' status), and a corresponding CWID column entry of the ... hmmm... it would have to be the actual QID of the CALLWKD station now that I think about it, to be able to go directly there to check, right?

FYI, I chucked in 3 new columns in the scores log just for my own interest, SCOREMAX and ACCURACY as per my last post, and BONUSES is a count of the number of bonus stations worked. its interesting to see the accuracy trend. I will remove if no use to you, but you might be interested with the result for now.

Quote

So another idea came to mind: QID2 (or 'Son of QID' :^) If we added one more CWID (Call Worked ID) column to the far right of the Error log, this could accomplish the mission like with the CID. Of course, a CWID number would only display IF the Error line involved a CALLWKD by another log submitter. Yes, this would be Awesome(2) and save even more time.

I think I understand what you mean, currently the qid represents which line the error triggered on. A cwid would be the qid of the call worked. If so, this wouldn't work how you think, since a call might have multiple qso entries for the sign, name and qth combination, therefore a single qid couldn't be derived.

Quote

This is still baffling me a bit. I understand 1 and 0, but for example:

Quote 427 K6NV VE3KI 80M 0242 RICH ON CNQ 1<3 -

VE3KI did not submit a log, but a determination cannot be made as to whether the QSO is actually 'invalid' or not until investigated.

phase2, the automation phase, decided it was invalid. If you had adjusted its weight before phase2 appropriately, then the outcome could have been different. OR, you could have run phase2, analysed the automated outcome, adjusted its wtf, then re-run phase2. Could you confirm that this makes sense, its important you understand the purposes of phase1 and phase2. This brings me back to the point of perhaps constructing more detailed logs at phase1 if the weights log doesn't provide what you need as of yet. Fundamentally though, you should have used the weights log to decipher that "VE3KI RICH ON" is valid, you can't rely on the outcome of phase2 until the phase1 weights log is acceptable, or appropriate adjustments in the adjustments log have been made.

Finally, I did mention I changed 1, 0 and - to appropriate labels of submitted, unsubmitted and invalid, but have replaced with LOG for 1 / submitted only and other "statuses" are left blank.

Getting down to the tax deadline so will have to get back with you later or tomorrow regarding some of the items in your last posting.

I downloaded the latest version and did a real quick updated comparison of the scores. VERY interesting with the other data columns you added. Note that I put the word BEFORE in Bold Red Font on the top line :^)

http://www.xgenesis.com/hashorama/compare_lqpck5_13April2015.pdf

At a quick glance, most scores appear to now be in-the-zone, and differences are mostly the result between 'Mulligans' that were given for minor errors in 2014, and just spot-checking logs that were not contenders for any of the category awards. FYI, Cat #11 is 'Checklog' only, and along with #12 & #13 not eligible for any awards.

Even without any Adjustments being possibly made from scrutiny of the Error log, FORTUNATELY even at a first pass, the individual category winners remain the same {Major SIGH Of Relief}. The sequences are a bit out of order in some cases, but for quick-glance purposes, I thought you would find this update very interesting.

BTW, I discovered one more (non-critical) left-over item in the entries.txt file ... an extraneous "CA" to the right of QSO #1257 after WQ5L RAY MS ... now deleted in my file here.

Ohhh... I did just notice something odd with this one you might want to check out:

Code

2 N8XX 11000 12000 92% 0 JACK MI

In my 2014 RESULTS, I showed N8XX at 13,000 points - his log had 13 QSOs (x 1000 = 13,000) "Claimed", which *should* have been the MAXimum possible score before any Errors in the scores.txt file list. Not sure why only 12,000 shows as the MAX, unless it has to do with him using the name 'IGOR' for the 1st QSO. IMHO, that should not make any difference in determining MAXimum possible.

Thanks for posting the comparison, its looking better than a few comparisons ago.

I couldn't locate the lqpck5 subdirectory, could you post the full link.

Quote

In my 2014 RESULTS, I showed N8XX at 13,000 points - his log had 13 QSOs (x 1000 = 13,000) "Claimed", which *should* have been the MAXimum possible score before any Errors in the scores.txt file list. Not sure why only 12,000 shows as the MAX, unless it has to do with him using the name 'IGOR' for the 1st QSO. IMHO, that should not make any difference in determining MAXimum possible.

You should have increased the wtf in the weights log for igor to above the wtf threshold. Ok lets simply this, lets remove one of the more extreme checks (which this issue tripped up on) and assume all the log side of things as oppose to call are somewhat valid and not ignore logs below the wtf threshold. Inevitably you can still control the validity on a per error basis via the adjustmenets log.

Delete or comment out line 522, which should be:

Code

next if $log_wtf < $wtf_threshold;

This may fix numerous issues where previously you hadn't used the weights log to its full potential. Note igor will now appear in the list of names in the unsubmitted and scores logs (this can be changed if need be), but any calls to this name will continue to CNQ error because its < wtf threshold.

One more tidbit... I pulled this info from an LCR (Log Checking Report) from a Contest I did operating from the U.S. Virgin Islands in 2004. I was young then (60) and operated 44 out of the total 48 hour contest weekend.

After being off the air for about 30 years, I only had about 2 months to get my Morse Code speed back up to 35 to 38 Word Per Minute to compete in this 'International' event. Ended up in the Top 10 :^) No way at 71 now would I attempt to do another 44 hour thing - I started hallucinating at about 38 hours and that was 11 years ago.

With 4603 raw (Gross) QSOs, and copying exchanges faster than I could type them, I wasn't sure how things would end up.

This will give you an idea of how one of the 'Major' HAM Contest sponsors deals with log-checking. Not sure the columns will align properly.

Made the change and yes, the scores.txt file updated but not the unsubmitted.txt file. Re-ran n=1 and n=2 twice but no dice.

In Reply To What difference were you expecting in the unsubmitted log, if with regards to n8xx, they submitted a log therefore will not appear there.

Not sure exactly what you mean here. Was just getting ready to post something for you, so will have to re-read yours once or twice and go look at the files :^)

Assumably you thought the unsubmitted log should have changed when you deleted the line of code. I was wondering what change you were expecting. I didn't expect it to change since this code deletion is seperate to the algorithm that determines unsubmitters.

I see that you are aware the qwid cannot be straight forwardly looked up for nil and qth errors. The time thing might work out, you could have a configurable range and then provide the range of qids i.e. cwid = 1001-1007, atleast giving you a rough position. You mentioned both sides of an error transaction should show up in the error log, I can't be certain of exactly what you mean, but remember if someone didn't return the call then that person won't have an error marked since they didn't log it but as a result won't get any points for it anyway.

Cool to see how the results were laid out for a contest you took part in. I like the summary sheet they provided you with. We could create a directory called summaries, and create a summary file for each log sign, with the same sort of layout but could extend it to display their entries from the entries log and errors from the errors log, maybe even their weights too. This is probably of low priority and something we can think about later.

I've gradually become more concerned that parts of what I've implemented aren't exactly what you thought I implemented (mostly with regards to weighting). Inevitably as and when I wrote sections of the code, I used my own initiative to fill in the gaps, I've tried to explain some of these, but ideally you would be able to read back and understand the code. Basically there are quite a few issues that you raise that from my point of view are working as expected. We could go back to using a fresh entries.txt file which has a much smaller sample of data that reflects every possible scenario, with a manually written expected output, making sure we are both on the same track. Maybe I am in actual fact over thinking this and that you fully understand and are perfectly happy with the way its going so far. Its been difficult for both of us to account for all the discrepencies along the way, this is not a straight forward task and its been good practice!

Sorry...was intensely focused on making the tax deadline yesterday. Mission accomplished, but it left me drained. I 'crashed and burned' last night ;-(

Yes, this is not a run-of-the-mill project/task scenario, and I greatly appreciate you hanging in there with me !!!

Today I started to take a new 'Test Drive' with this year's 2015 data, but ran into a few challenges that necessitate re-downloading and re-formatting some of the Callsign/Category data (the QSO line stuff appears to be OK).

Unfortunately, I need to get all packed up to leave in the morning for an Annual Convention so will be offline until Monday. I plan to use the (normally boring highway) 3 hour drive each way to periodically 'chew' on final decision options as to strictness or leniency in dealing with errors in the script processing. I'll have my weenie digital recorder along to take notes. Hmmm, this will likely be on the way back, as I think I need 3 hours of non-stop music on the way 1st leg of the trip to further de-compress :^)

I'm actually looking forward to several days of No Confuzzzzer, No email, No Robo-Telemarking phone calls, etc. Hopefully I will return with an emptied personal mind-cache to meet the remaining 'challenges' in this matter with a fresh perspective.

I'll be PM'ing you more specific details hopefully later today, but am uploading a quick "Bird's Eye View" here of what came to mind before leaving for the Convention and subsequent tweaking and re-tweaking done since returning.

Out of the Total 101 errors using WTF=3, this can reduce the "iffy" (questionable) CNQ status count to only 35 !!! With a future increase in overall log submissions and QSO counts, this potential time savings is worth-it's-WEIGHT-in-Gold ('Weight' PUN intentional :^)

The new "Discovery" on how to further reduce the manual PITA of switching between files when investigating and researching errors came about when I switched from using the 1024x768 display main Notebook Confuzzzzer to a seeing things much w-I--d---e----r without having to horizontally scroll on the 23" 1920x1080 flat panel with a recently acquired Desktop for A/V purposes. What a difference...a whole new World!

I've embedded some brief "NOTES" in new consolidated errorlog.csv file approach for quick import into Excel (.csv instead of a Tab/.txt file is necessary for this to work). All the other existing errors.txt, weights.txt, etc. files will still serve a useful purpose.

Oh darn... the .jpg won't upload (exceeds 250KB). Hold on... OK, you can get it here:

Code

www.xgenesis.com/hashorama/errorlog_partial_screencapture_1920px.jpg

(EDIT): I think that should be sorted DESC instead of ASC for the CNQ Weight/CNQ combinations in the Notes (for the CALLWKD LOG CNQ WEIGHTS column/field entries) ;-(

I've made the changes you described, but as a result I ideally need to refactor the code then re-test. Basically the new error sort order required an intermediate data structure therefore am considering putting error and score construction in their own "_input" functions.