Actions from fuzzixMovable Type Pro 4.382014-01-26T02:39:27Zhttp://blogs.perl.org/mt/mt-cp.fcgi?__mode=feed&_type=actions&blog_id=0&id=1988Commented on blogs.perl.org security breach in Metatag:blogs.perl.org,2014:/users/meta//1290.5577#13149492014-01-26T02:39:27Zfuzzix
Thanks to all involved for prompt notification and professional handling of the situation.

Is there a current project aimed at replacing the MT install? I'd like to register my involvement even if it happens I have nothing useful to contribute... :)

]]>
Posted Repliconz - dev log 0 (Perl and SDL) to fuzzixtag:blogs.perl.org,2014:/users/fuzzix//1465.55782014-01-25T14:37:08Z2014-01-25T14:51:42ZI posted this yesterday on my personal site and realised it might just act as a reasonable "getting started" type article for SDL with Perl. Further articles in the series probably won't have this level of detail, so I'll keep...fuzzixhttp://blogs.perl.org/mt/mt-cp.fcgi?__mode=view&blog_id=1465&id=1988
I posted this yesterday on my personal site and realised it might just act as a reasonable "getting started" type article for SDL with Perl. Further articles in the series probably won't have this level of detail, so I'll keep them on my own site. Anyway, on with the show.

So I decided to play around with Perl's SDL bindings.
A game appears to be happening, so let's play around with this dev log idea too,
documenting the process and pitfalls of making some stuff move around on screen.

It turns out you don't need to know a whole lot to make this happen, in the
simplest cases at least. Let's see what we have so far:

OK, so we're not going to set the world alight just yet. Anyway, we have a guy
and some bullets. We are missing enemies, scoring, action, pew pew noises and
any incentive to play. These come later, I hope.

Perl's SDL distribution comes with a convenient extension called SDLx::App
which is built on SDLx::Controller.
This makes creating game loops simple. You set a delay between iterations,
handler callbacks for the things a game loop usually contains (input events,
drawing and so on) and then run it. Each callback gets a delta-T value (the time
that has passed since the previous iteration) so you can tie this value to your
physics model and get consistent movement even if you are occasionally starved of
resources or change your frame rate (delay).

The three classes of handler we need to define are event, move and show. For
those of you familiar with MVC style frameworks, these handlers are (very)
roughly analogous: Model ~= move, View ~= show, Controller ~= event.

It should
be noted that any number of callbacks for each handler type can be added, though
this code currently has just one for each.

Event processing is triggered on input, mouse movements, container changes and
so on. Our callback function receives an SDL::Event
instance containing the queued events which need handling. Let's take a look at the
event handler in Repliconz:

There's a little more going on here. First we go through the keys which were set
in the event handler and add 1 (since we end up manipulating this unit vector
with an actual velocity later) to X/Y velocity for each control. Simply setting
them to 1/-1 means that the hero will still move when opposite controls are being
pressed, so we add them to prevent this: 1 and -1 being added will cancel each
other out. Ignore the $bomb stuff, that's in the "notion" stage of development.

$self->{hero} is an instance of Game::Repliconz::Guy. Let's take a look at its
move method:

The moving itself is taken care of by setting our new coords to our unit vector coords
multiplied by our velocity and delta-T, the time that has passed. This is the key to
getting consistent movement even if timing / framerate changes. You can check
this out by reducing the delay in app setup to effectively increase framerate.
You can also manipulate $dt in your callbacks to speed up or slow down the action.
For example, to add a slo-mo mode you could divide $dt by 2.

The last four lines reset our position if the guy tries to move outside the bounds
of the play field. What is constrain_velocity_xy up to?

There are a couple of things going on here. The first is we reset the bounds of
our unit vector set in the move handler callback. Since there are two sets of
controls and we used an 'additive' approach to setting the movement, if you press
two buttons for the same direction, you end up with twice the required vector
length and, eventually, twice the velocity. If any direction is set, we reset it
to 1 or -1.

The other thing is, if we are moving diagonally, our vector is no longer length 1,
it's the length required to go from corner to corner of a square with sides of
length 1. Pythagoras tells us this is ~1.41. To correct this and set our vector
length back to 1, we reduce the coords of our vector end point to ~0.7 (sine of 45°)
of their original value.
This diagram on Mathematics For Blondes
illustrates quite effectively why this works, I think.

If you watched the video above, you saw that the guy wasn't the only thing moving.
We also have bullets. Bullets are instances of Game::Repliconz::Bullet, which are
pushed in and out of a queue (currently arbitrarily limited to 20 items) in our
instance of Game::Repliconz::Guy when shoot is triggered by the fire button:

We track time elapsed by adding delta-T values until our gun cooling time is reached.
This allows us to easily control rate-of-fire at any stage, to implement a high ROF
bonus by just reducing the cooling time, for example.

We tell the bullet about the guy and mouse position, but how do bullets aim in the
mouse cursor direction? Another unit vector is created on instantiation, based on the
guy's position and the mouse cursor position:

So we set the initial x/y of our bullet to be the centre of the guy, along with
width and height for drawing later. We then make use of Pythagoras' theorem again
to convert the difference between the start point and mouse position into a unit
vector. If you consider the X and Y axes of our coords to be the sides of a
right triangle, adding their squares will give us the length of the vector guy ->
cursor. We just need to divide our coords by this value to wind up with a vector
of length 1.

Once you have all this information, the move method becomes very simple:

]]>
Commented on Fun With Fuse in fuzzixtag:blogs.perl.org,2012:/users/fuzzix//1465.3736#2129212012-09-25T09:09:16Zfuzzix
Thanks, glad you found it useful.

A thought occurs, that instead of all the binmode/read/handle stuff, you might be able to 'use bytes;' and then use plain substr. That might simplify things, if it worked.

]]>
Commented on Perl, PHP, Python, Ruby Job Trends on Indeed.com in Bryan Gmyrektag:blogs.perl.org,2012:/users/bryan_gmyrek//1508.3816#2109262012-09-17T10:01:25Zfuzzix
I don't think I can take plain numbers of job postings seriously.

Anecdata, but in .ie it is common for recruitment agencies to post fake ads just to get you on the books. I would imagine they use lists like tiobe to cast as wide a net as possible. Some postings may be real, but I was left scratching my head when I was mailed a spec looking for experience with the "agile waterfall"

How is this not illegal? No idea, I think the only laws here governing job postings refer to equality.

]]>
Commented on How to Post an Article on blogs.perl.org in shawnhcoreytag:blogs.perl.org,2012:/users/shawnhcorey//102.3826#2095412012-09-13T14:38:35Zfuzzix
Thanks for this, wasn't immediately obvious to me.]]>
Posted Fun With Fuse to fuzzixtag:blogs.perl.org,2012:/users/fuzzix//1465.37362012-08-24T15:13:40Z2012-11-07T08:10:45ZFUSE (Filesystem in USErspace) is a useful kernel module with an API which allows file systems to be implemented in user space applications, mounted and fully integrated into the system's VFS. Originally implemented for Linux, FUSE API-compatible kernel modules are...fuzzixhttp://blogs.perl.org/mt/mt-cp.fcgi?__mode=view&blog_id=1465&id=1988FUSE (Filesystem in USErspace) is a useful kernel module with an API which allows file systems to be implemented in user space applications, mounted and fully integrated into the system's VFS. Originally implemented for Linux, FUSE API-compatible kernel modules are now available on *BSD, OpenSolaris and MacOS. That said, this was written on a Linux machine so there may be assumptions made about the tools available, their output etc. Feel free to submit info on other *nix-like systems for inclusion if you encounter any inconsistencies. This also goes for corrections and suggestions - if I get the details plain wrong or the code stinks and so on...

Some examples of popular FUSE file systems are ntfs-3g, which provides full NTFS support for the Linux kernel, and sshfs, which mounts remote directories over SSH with SFTP support. FUSE also powers ambitious projects, such as the distributed filesystem, MooseFS as well as providing easy access to proprietary protocols such as MTP.

We can combine this power with the power of Perl using the excellent FUSE API binding.

]]>
In this post we will implement a simple FUSE module to read an RSS feed right in the file system. Why an RSS feed? It's a handy source of data to populate files and demonstrates nicely (I hope) how Perl and FUSE can work together. I will use FUSE here to refer to the API and Fuse to refer to the Perl module / bindings.

FUSE works via callbacks to perform the basic operations of a file system, such as fetching a directory listing or retrieving a file's attributes (type, permissions etc.). These are implemented in the Fuse module by passing coderefs in the arguments of the call to the main loop.

A minimal example of this is the following code to create directory listings which consist of the English alphabet. It implements two functions of a file system, getdir (return a directory listing) and getattr (retrieve attributes for a file/directory). References to these functions are passed in the call to Fuse::main - when fuse needs a directory listing or file attributes (type, owner, permissions, size...) it will call these coderefs.

So, getdir returns an array, 'a'..'z', but why is the last element 0? The last element of the returned array should be an errno, 0 indicating success. Some negative number usually denotes failure. We will look at cases of returning a failure status in further examples.

What's going on in getattr? For this simple example a set of default values for the directory entry's attributes have been used. We can see the effect this has on the listing when we actually use the module. Yes, we can mount this small module and examine its contents. Let's take a look at what it does:

We can see the directory listing consists of the array returned from getdir. Where does getattr come in? For each entry in the listing we must return attributes so the shell knows how to present it, access can be granted/denied and countless other things. So, what do some of the values we returned from getattr mean?

We don't need to worry about most of these for now (for a more complete description, see the Fuse or stat() documentation), but we can observe how the values returned effect what we see in the listing, like so:

$ ls -ld listing/a
drwx------ 1 root root 0 Jan 1 1970 listing/a

The first column we see is drwx------ - the type and permissions (or mode) of the entry. We passed 0040700 for this. The 0040 part describes the type - directory. If this were a file, this value would be 0100. The second part is probably familiar to anyone used to Linux file permissions, it grants read, write and execute (or enter for directories) permission to the entry's owner.

Next is the link count, nlink, which counts the number of hard links to that file or directory. Its entry in directory listings counts as a link, hard links created with ln count as a link... The link count can be used by tools such as find to make decisions about recursion. We set this to 1 - there is always at least one link. This feature is where unlink() takes its name from, we are not deleting files as such, we are deleting names which link to a file or directory. The system reclaims the file's space when the last link has been removed.

The next values we see are the uid and gid - we returned each of these as 0 so the directory is owned by root:root.

Next are the time stamps, access time, modify time, create time. Each of these is epoch time, or the number of seconds since 00:00 on 1st Jan 1970. Since we return 0 for these, the directory appears to be many years old.

The last column is the directory name. Since we return the same attributes for any getattr request, all the directories will appear much the same. Also, since we return the same set of contents for each getdir callback request we should have a recursively alphabetic directory tree...

The user_id and group_id are a component of FUSE's security model - unless explicitly granted, FUSE mounts belonging to you are not accessible to any other user, including root.

To unmount the filesystem we use the fusermount tool which should ship with FUSE:

$ fusermount -u listing
[1]+ Done ./listing.pl

Running mount again should confirm that the module has indeed been unmounted.

So far we have demonstrated a functioning FUSE module, though its utility is limited. We can remedy this with the magic of data! A reasonably dynamic source of data is a RSS feed for a busy website. We will use a hash populated with a file system structure and the content of an RSS feed to demonstrate how easily a data set which fits naturally into Perl's capabilities can be presented using the hierarchical file system abstraction (though this particular example won't demonstrate a hierarchy so much).

How will we bend the data from RSS to a file system layout? A simple mapping of the usual RSS feed elements should help; each file's content is a single post, the file name is the title, the file's time stamp is the post date and time... you see where I'm going.

The first task is to populate the hash from an Atom or RSS URL, we can use the XML::FeedPP module to achieve this fairly trivially. The file content itself might just be a dump of the post's content as-is, but for the purposes of this we'll convert it to plain text. HTML::FormatText::WithLinks should prove useful here, providing a readable format while keeping links intact. The post title itself can contain HTML markup and entities which could make the file name difficult to read or even nonsensical - HTML::Strip should clean it up nicely. Retrieving an epoch time stamp from UTC or other formatted time stamps is trivially achieved (from our perspective) with Date::Parse.

So now we have all the components to turn the RSS feed into a file system, we can proceed. The following function takes a preconfigured feed URL ($source) and chops its components into elements of a hash with file scope containing the file system contents (%files).

The function starts by adding '.' and '..' to the listing of files - you may have noticed that these seemingly perennial entries were missing from the full directory listings in our first example. They must be managed by the FUSE module explicitly.

Other than '.' and '..', we do not create any other directory entries here - in order to keep things simple (in functions such as rss_getdir()) we'll just have a plain dump of articles into files. To include subdirectories would require a slightly different structure, perhaps using the 'content' key to describe lists of other files and directories. There is no need to specify attributes for '..' here - as '..' refers to another file system in this case, the attributes will be retrieved from that file system module (most likely the file system driver for your home directory).

We then iterate through the list of posts, or items, that XML::FeedPP parsed from our feed and create a new file for each entry. The file name is set from the item's title which has any HTML stripped and any occurrence of the '/' character replaced - POSIX restricts use of '/' to the directory separator so it is not allowed in file names.

The file's contents are set to the item's "description" - the description is generally specified as a field to provide a short description for the provided link, though many feeds place the entire article in here. The original link is then appended to the text.

We now have our hash, populated from the parsed feed. How do we go about presenting it? We saw earlier that the minimal set of implemented callbacks for a working file system amount to getdir and getattr, so how about we continue by implementing them. Retrieving a directory listing is trivial enough, we simply return a set of the entries created in populate().

sub rss_getdir {
return ( keys %files, 0 );
}

To implement getattr we mix some of the values we set in populate() with values taken from the file's entry in the %files hash, Fuse as well as some hard-coded defaults.

The path_to_ref function returns a reference to the element in %files which matches the requested path by simply removing the initial directory separator returning that named key. Since a request for top-level directory is '/', this must be caught and set to the actual entry we created for the directory in %files, '.' . This function is used by getattr and open.

The rss_getattr() function's use of path_to_ref() can trap non-existent files trivially. The file size is set to the number of characters in the file, rather than the number of bytes, which would usually be what a file system module returns - this is good enough for our purposes here. The mode is made up of a bit-shifted type added to the mode flags so a directory type (0040) with mode rwxr-xr-x (755) permissions is returned as 0040755. User and group ID values are pulled from Fuse::fuse_get_context. Timestamps are all set to the post's time we stored in populate().

So far we have just enough to download a feed and list its contents in the file system. How do we go about reading the posts? We need to implement just a couple more callbacks to make this happen. The first is open. Since we don't need to generate handles or any other low-level work, all we need do is confirm the requested file exists and is not a directory...

...Well, that would be the case, if we still lived in an 8-bit world - we could use substr in the read callback to return the number of bytes from the location requested. We have already "cheated" in this file system by not returning the size of the content in bytes, but characters. For read, we need to return the actual set of bytes requested, regardless of string encoding.

We can take advantage of a feature in Perl, the in-memory file handle, to get real bytes from our file's content, ignoring the character encoding. If we return a file handle from the open call, subsequent read calls will get that file handle to work on. So, let's try that again:

The (almost) final callback we need to implement for a working file system is read. The parameters passed to the read callback are the file handle (since we returned one from rss_open()), offset (the byte to start reading from) and the number of bytes to return.

When we mounted our first demo file system, we had to background it on the command line with '&'. To have the file system background itself, add the following line before the call to the Fuse main loop:

fork and exit;

I think we're ready to try the module now. There are some small pieces I didn't cover here (retrieving command line parameters, the exact form of module includes...) so you can retrieve the full source.

Now to locate a RSS/Atom feed - I hear good things about blogs.perl.org, so...

While using Devel::NYTProf on a new application I started getting this
message
. . .

...Well, the module appears to be working. One last thought occurs to me, what if I want to obsessively update the feed to stay right on top of what's new in the Perl community? Well, we could unmount and remount the file system to run populate() again, or we could add some hook which runs it for us.

It's arbitrary enough, but we could add functionality to rss_open to trap, say, `touch update` to repopulate the file system hash. We'll need to add this to the initial set of files in populate() as we have no facility to dynamically create arbitrary file system entries: