Introduction

This article presents a viewer for CodeProject's recently introduced reputation information. Completely futile, but fun to create, and fun to have available.

Overall Functionality

The first view presents information on a single account; it shows the user name (HTML rendered in a little WebBrowser), the reputation graph, and a list of articles (real ones and Tip&Trick ones).

The second view presents a list of "highest achievers", i.e. the people that show up on the Who's Who
pages when querying by reputation, by article count or by message count. Values for all reputation categories are listed here.

The third view offers counter values for the same members.

Some Functional Details

Some effort has been spent to offer functionality together with user comfort:

The User ID, when changed, gets stored in the registry; a single registry key is used under CurrentUser\Software\CPVanity\MemberID, so the utility will start by showing the last account it has presented in a previous session; unless it gets launched while holding the control key (no account shown), or a userID is passed on the command line (Yes, you could make desktop shortcuts for a number of accounts).

To honor CodeProject conventions, hitting CTRL/F5 will refresh the current view.

The champion for each reputation category gets a blinking red background, and the one account (if any) gets a blinking green background when present amongst the highest achievers. This blinking effect is conditional on a checkbox and gets persisted in another registry key CurrentUser\Software\CPVanity\Blinking.

The "Highest Achievers" page offers hyperlinks to the profile page of all the listed accounts;
when clicked with control key down, CP Vanity loads the "One Account"
view with the selected account.
Since version 2.3 the "shift" button (as well as the "shift" key!) scroll the DataGridView horizontally
and reveal several new values, the total counts for articles, tips, blogs, messages, questions, answers, and comments.

Limitations

There are a lot of limitations to CP Vanity, some by design, some by the principles that had to be used in order to collect the necessary information. Here is a list:

The window width is fixed at 1024; that is the width the DataGridViews need to display all their information; since I don't like horizontal scrolling, I decided to settle for a fixed width that should be no problem except for the smallest displays.

CP Vanity shows a snapshot; it does not provide a live image or an automatic refresh.

The information is gathered by fetching a couple of CodeProject web pages and scraping their HTML content; there is an ever increasing risk that something will change (the URLs used, the page content, the HTML tags used, whatever) to such an extent that CP Vanity no longer functions as intended. If so, I may or may not provide an update.

The Highest Achievers list is aggregated from the Who's Who pages, which basically offers the
top members ordered by total reputation, by article count, or by message count. For some of the reputation categories,
many points could be earned without creating lots of articles or messages (e.g. Organiser points are earned by voting),
so it is conceivable, but not very likely, that some highly ranked people will not show up on the "Highest Achievers" page.

Some Technical Details

All the CodeProject specific things are isolated in the CPSite class, which offers methods to get URL strings, to fetch web page content, and to extract the required information. If anything changes at CodeProject, CPSite is where the changes may have to be reflected.

All web page fetching and scraping is performed by a BackgroundWorker, and of course the GUI updating is handled in the ProgressChanged and RunWorkerCompleted event handlers.

The TabControl Tabs

The configurability of the WinForms TabControl is very limited; all the tabs are rendered identically, using the TabControl's Font, ForeColor and BackColor values. So a different look-and-feel (multiple colors, hovering effects, ...) are not supported at all. This often is the subject of a question in the programming forums.

As I wanted a menu bar rather than the typical tabs, my dilemma was to either use a TabControl and somehow fake a tab strip, or use superimposed Panels rather than TabPages and control their visibility myself. The one advantage of the TabControl is that one can design the TabPages in site, within the Form itself; no separate Forms or UserControls are needed. So I decided to go for the former, and take on the fight with TabControl. This is what I came up with:

Inside Visual Designer, I added a Panel and a TabControl to the Form. The Right and Width properties match, the Panel.Height is sufficient to hold the new tab strip and to hide the unwanted tabs (once the Panel and TabControl get moved over one another); however the TabControl's position is under the Panel, so the TabControl initially is a bit lower, allowing both of them to be designed with ease.

At run-time (i.e. in the Form constructor, right after calling InitializeComponents), the TabControl's Top gets reduced and its Height gets increased by the height of the Panel, effectively sliding the native tabs under the artificial tab strip. Et voila, the ugly tabs are gone, and the look-and-feel is what was hoped for. In reality, the tab strip isn't as tall as the old tabs, however a second panel above them hides whatever protrudes.

The left image shows the Designer view: the new tab strip (green panel) will be used to hide the original (gray) tabs, which have Appearance=FlatButtons. The right image shows the run-time view, the original tabs are hidden by sliding the TabControl up and under the other panel(s).

DataGridView Problems

I've had some trouble getting the DataGridView controls to correctly render empty cells! Everything works fine when background coloring is not required; however, when a cell changes from non-empty with background color into empty (always without background), then the cell does not get erased.

The one workaround that solved (most of) the unwanted mispaints, consisted of three special measures:

When the TabControl switches to another page, the newly visible DataGridView is made invisible for a while with a simple Thread.Sleep(100). That is sufficient to get a clean background before the control gets rendered as required.

When the application returns from the background, the same repaint is forced.

When the DataGridView gets sorted, the background coloring is suspended for a while; i.e. a boolean flag controlling background colors is set false, and a timer is launched to set it true again some 100 milliseconds later. Without this trick, toggling the sort order in some columns creates a mirrored image, where the cells from the top and those from the bottom look identical (whereas one set of them should be empty).

The above did not solve the problems about repainting (or lack thereof) that should occur when the DataGridView gets scrolled, so I kept looking for a real solution. And I finally discovered what seems to be the real problem: empty cells only work properly when no background color gets set (not even White or Transparent), so the CellFormatting event handler has to check the reputation points before setting the BackColor, and skip all of it for a zero value. I fixed the code accordingly in version V1.3 and removed the code related to the earlier workarounds. All is well now.

Regex Performance

While I'm no Regex expert, I wanted to come up with patterns that would be somewhat tolerant to changes in the HTML code used. Initially that resulted in some complex patterns containing multiple wildcards, some greedy, some lazy. The net result was scraping tended to use several seconds per page.

In a second iteration, I simplified the Regex patterns, used fewer wildcards, and fewer groups; and I used some simple string searches too. As an example, the user names and the reputation numbers are extracted with regex patterns, whereas the colors are determined by searching separately for the color names. The speed up was spectacular, scraping time now is irrelevant.

Networking

To support situations where the PC is located behind a firewall, the configuration file got adapted to this:

When a proxy server uses password-based authentication, a NetworkCredential must be issued; look into file CPSite.cs and adapt method CreateWebRequest() as required.

Conclusion

CP Vanity offers two pages of highly-condensed information at your finger tips, most of it presented in DataGridViews that can be sorted in many ways. One remarkable outcome is that the seven reputation categories currently have six different champions.

Acknowledgements

Thanks to:

Sacha Barber, whose Really Vain Web Spider article provided a lot of inspiration, and from whom I borrowed an icon

Pete O'Hanlon and Don Kackman, for providing the firewall/proxy suggestions

All early users for reporting their findings and offering suggestions and comments

Version 2.0 (06-Dec-2010): added support for Who's Who queried by Reputation, and cbFetchJob ComboBox; added PlatinumCount column; several minor GUI improvements

Version 2.1 (25-Dec-2010): updated to cope with the latest web site changes

Version 2.2 (27-Mar-2011): added downloads column to articles table

Version 2.3 (24-May-2011): added several counters to the top achievers tab, see "shift" button;
updated and added some screen shots; added an executable-only download;
fixed a scraping bug (the "One Account" tab failed to deal with the thousand separator in article votes,
bookmarks, and downloads)

Share

About the Author

I am an engineer with a background in electronics, software and mathematics.

I develop technical software, both for embedded systems and for desktop equipment. This includes operating systems, communication software, local networks, image processing, machine control, automation, etc.

I have been using all kinds of microcontrollers and microprocessors (Intel 4004/8080/8051/80386/Pentium, Motorola 680x/680x0/ColdFire/PowerPC, Microchip PIC, Altera NIOS, and many more), lots of programming languages (all relevant assemblers, Fortran, Basic, C, Java, C#, and many more), and different operating systems (both proprietary and commercial).

For desktop applications and general development tools I have been using both UNIX systems and Mac/MacOS for many years, but I have switched to x86-based PCs with Windows, Visual Studio and the .NET Framework several years ago.

I specialize in:
- cross-platform development (making software that runs on diverse hardware/OS combinations)
- instruction set simulation
- improving software performance, i.e. making sure the software runs the job at hand in as short a time as possible on the given hardware. This entails algorithm selection, implementation design, accurate measurements, code optimisation, and sometimes implementing virtual machines, applying SIMD technology (such as MMX/SSE), and more.

I read CodeProject has yet again changed the way they HTML-encode article lists, and that is why CP Vanity isn't able to list them any longer. A Web Service is the only real solution, and CP may or may not provide that in an unspecified time period.

Sorry, I am not going to adapt CP Vanity any time soon; hunting moving targets is not what I fancy right now. If you want, you can figure out what has changed and try and adapt the relevant method(s); all the source files are available in the download.

The quality and detail of your question reflects on the effectiveness of the help you are likely to get.Please use <PRE> tags for code snippets, they improve readability.CP Vanity has been updated to V2.4

Sorry CP Vanity showed up on the home page again, nothing has been added or changed, except for an obsolete ZIP file that was removed from the associated files (it did show up under "Browse Code", potentially confusing the reader, so I had to remove it).

The quality and detail of your question reflects on the effectiveness of the help you are likely to get.Please use <PRE> tags for code snippets, they improve readability.CP Vanity has been updated to V2.3

if you mean: can one obtain the Who's Who ranked by a single rep component, the answer alas is no. That is why, when sorting by a rep component in CP Vanity, you may be missing a few high-ranked members. Fictitious example: someone with few or no articles, few or no messages, one rather high rep component in a not so popular category (say 10,000 enquirer points), but nothing much in the other components, hence total rep about 10K too, such a member would not show at all in CP Vanity.

If CP were to provide ranking by each of the components, CP Vanity could be made to get say the top-50 of each category, merge them all (as is done now already for rank by articles, by message count, and by total rep), then show a much more accurate list of top achievers.

The quality and detail of your question reflects on the effectiveness of the help you are likely to get.Please use <PRE> tags for code snippets, they improve readability.CP Vanity has been updated to V2.3

The quality and detail of your question reflects on the effectiveness of the help you are likely to get.Please use <PRE> tags for code snippets, they improve readability.CP Vanity has been updated to V2.3

The quality and detail of your question reflects on the effectiveness of the help you are likely to get.Please use <PRE> tags for code snippets, they improve readability.CP Vanity has been updated to V2.3

Just an FYI, this doesn't work when I try it from my work computer. It's probably a proxy/firewall issue or something, but thought I'd let you know. When I enter my member ID under "One Account" and click "Go", I get the message:

Could not access
http://www.codeproject.com
mid=3750834

It really wraps like that, and I'm sure there's more to the URL but it gets cut off. Also, when I click "Go" on the "Highest Achievers" section, nothing happens. I just see a blank page.

The executable-only ZIP didn't have the config file, I fixed that, maybe it solves your problem.
If not, there are a couple of things about proxies in the article. Did you have to patch the eerlier code to get it to work? if not, it should still work.

The quality and detail of your question reflects on the effectiveness of the help you are likely to get.Please use <PRE> tags for code snippets, they preserve indentation an improve readability.CP Vanity has been updated to V2.3

Sorry to say but for all versions up to and including V2.2 the article-oriented table on the first tab page shows erroneous counts for votes, bookmarks, and downloads whenever the actual individual numbers exceed 999 (the web scraping did not recognize the thousand separator).

I can't really comment on the exact situation (it is changing all the time), however your account shows a 10-point difference between tabular and graphical numbers. All CP Vanity does is fetching some of the Who's Who pages and scrape them.

here are some observations, you may want to take this to the suggestions&bugs forum:

1. when logged in, I can see your article list on CodeProject; it says "52 articles" and actually shows 54 of them, including "Zeta Resource Editor" and at the very bottom "Zeta Producer Desktop 8", all with an average rating of 4.43

2. when not logged in, I can also see your article list on CodeProject; now it says "52 articles" and shows 52, not including the two I mentioned above; the average rating now is 4.41

CP Vanity does not log in for you (it does not know your account name or password). So it reflects what is visible in (2).
I understand you would like it to show (1), but that is impossible, unless you somehow get (1) and (2) to fully agree.

I did not investigate why (1) and (2) are different; the only reason for a difference that I am aware of, is when some articles are still not public, and the viewer is either the author himself, or sufficiently reputed to have viewing and editing rights. On the other hand, there could be a bug in CodeProject of course.

the Who's Who pages have been changed again. Now they no longer show member names (never seen an anonymous Who's Who before). This makes it impossible for CP Vanity to operate, unless a lot of inefficient page fetches would be added, which I won't do.

I do not plan a new release any time soon; I'm waiting for a Web Service now, as I have decided to abandon the chase of a randomly moving target.