Introducing: YUNPM (Y U NO play movie!) library name suggestions are welcome...

To put it mildly, Java movie playback is a joke, and waiting for JavaFX to become the defacto standard might take yet another decade. That's why I've decided to bite the bullet and make a 2nd attempt (after VLC failed) to make a minimalistic Java Video Player implementation, using an external process to decode the video and audio in a media file: this time using ffmpeg.

Read your earlier posts on this, but not able to respond as on holiday and posting on here with Android is horrible!

This seems an interesting approach. I wondered if you'd compared this to using GStreamer, and what you'd consider the pros and cons of either approach? Your approach must add some extra overhead (extra encode / decode, etc.), but presumably that's usable. Be interesting to benchmark them, particularly as the underlying codec code is most likely the same.

Most of the code to do this bundling and playback with GStreamer is hidden in this thread - I haven't had a chance to tidy it up further since helping Cero get it working, but can hopefully have another look sometime soon.

I'm not really interested in the most optimal solution. I thought it was easy to do it with VLC, then got stuck with VLC exporting 1 image-file per frame (I forced it to be a TGA file for performance) which ended up with the requirement of a ramdisk - which is not easily achieved on Windows. Then I decided to give ffmpeg a go, which was only able to generate either mjpeg or wav, so i decided to spawn two processes, streaming files to disk and reading it back in java - an hour later i realized that ffmpeg supported writing the data to stdout, so I didn't have to touch the disk.

Regarding gstreamer: it might work, it might work infinitely better, I don't know, I never used it.

My library is quick 'n dirty, but it's extremely easy to setup and run and reliable to the point that you have no way to break it (famous last words) and it handles syncing video and audio for you, which is a non-trivial problem for certain videos, as the number of audio samples per frame is not always an integer. I create (bruteforce) a pattern of swapping differently sized buffers, and usually get the sync-error below 0.01s per hour.

There is barely any API on the Java side (no seeking), but does it matter? Now we have a trivial way to playback video in Java, that's more than what we had in the last decade.

Maybe my library will spark enough interest in 'java video' for somebody to write something better. In the meantime, the getting-things-done approach prevails and everything else is premature optimization.

Feel free improve on it! (and provide a gstreamer backend, while you're at it )

Hi, appreciate more people! Σ ♥ = ¾Learn how to award medals... and work your way up the social rankings!

/* * Copyright (c) 2012, Riven * * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * * Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * Neither the name of Riven nor the names of its contributors may * be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */

My guess is that everybody wanted to write a 'proper' solution, instead of this quick hack. As a proper solution is a lot of work, nobody did it. I thank the JGO community for this insight... I never used to get anything done for the very same reason.

Hi, appreciate more people! Σ ♥ = ¾Learn how to award medals... and work your way up the social rankings!

Nothing prevents you from porting this API to LWJGL, most of the code relies on LibAV and FFMPEG but you would need to modify a bit the native windowing toolkit of LWJGL to support some Broadcom non standard (not EGL compliant) initialization routines.

There is barely any API on the Java side (no seeking), but does it matter? Now we have a trivial way to playback video in Java, that's more than what we had in the last decade.

Maybe my library will spark enough interest in 'java video' for somebody to write something better. In the meantime, the getting-things-done approach prevails and everything else is premature optimization.

Except that a number of 'proper' solutions have existed for a while! eg. GStreamer-Java has been around over 5 years. The only difficult thing until recently has been getting hold of the pre-built native libs for Windows & OSX, but that's no longer a problem. Decoding video and audio to bytebuffers is trivial as it's all 'in-process', so no need for writing to / from JPEG and GStreamer handles the sync for you.

I don't understand why with VLC you were using the write to file approach, either, rather than the VLCJ bindings which also give you direct bytebuffer access.

As an approach I find this interesting, but if the intention was 'getting-things-done', this seems like the long-winded way!

Why were people struggling with it so much (according to many recent threads), calling it a disgrace that Java was still lacking video, and coding (new) solutions, giving up, etc. And why are people throwing medals in my general direction?

So many questions! Maybe it's just lack of exposure of existing solutions, or horrific APIs.

Hi, appreciate more people! Σ ♥ = ¾Learn how to award medals... and work your way up the social rankings!

You cannot seek, you cannot adjust the volume. Basically, you cannot do anything, except watching the video, until it stops. I can easily add support for pause/resume and stop, but that'd be all.

IMO all you need for video game video playback is: play, stop(in case a user cancels), isDonePlaying (to proceed with different code afterwards) and setVolume would be nice in some way or else huge volume differences may occur when ingame volume is low and stuffpause and resume CAN be nice, when you have something like, press a button during a video -> "really skip video?" (while video is pausing)

As an approach I find this interesting, but if the intention was 'getting-things-done', this seems like the long-winded way!

Its all about something that works easily from the programmers view of things(Xuggler fails here), easy for the users view of things(it just works, no prev. requirements) and it has to be legal, meaning that all code and codecs have to be something like LGPL. (VLCJ is still GPL)

Technically I have video playback running with gStreamer, and its beautiful; however if this is equal in performance, then I could technically see me switching to this, because we include windows and mac libraries and no linux ones and "expect" with a very high degree of probability that gStreamer is already installed on linux machines.This lib would make that tiny risk vanish.

So many questions! Maybe it's just lack of exposure of existing solutions, or horrific APIs.

It was always something:

JMF - laughable, no further commentJavaFx - javafx1 had license problems and javafx2 seems to have them tooXuggler - "Very nice, however there is no player, and people who are much more skilled than I try to write a player still have sync problems."Cortado - Not reliable, skips and freezesJGStreamer - seems fine.VLCJ - GPL license

Why were people struggling with it so much (according to many recent threads), calling it a disgrace that Java was still lacking video, and coding (new) solutions, giving up, etc. And why are people throwing medals in my general direction?

Well, I wouldn't dare to claim that some people like to be spoonfed! Actually, it's probably mainly an exposure issue, and the fact that the solutions that have the most exposure are the least suitable. Having tried most of them, I'd claim GStreamer as probably the better option - VLC also good bar license issues.

The problems in most of those threads (half of which might have been Cero's ) were generally with obtaining and bundling native libs, and with difficulties knowing how to upload those video frames to OpenGL textures - both of which are easily solvable.

Technically I have video playback running with gStreamer, and its beautiful; however if this is equal in performance, then I could technically see me switching to this, because we include windows and mac libraries and no linux ones and "expect" with a very high degree of probability that gStreamer is already installed on linux machines.

Bearing in mind that GStreamer, VLC and FFMPEG are pretty much using the same code (libav is an FFMPEG fork), then I seriously doubt that the added indirection in Riven's solution could match the performance, but it may not be enough to be an issue. You could easily bundle the Linux libs in the same way you're already doing mind you (just cos I said I had no interest in doing it doesn't mean you shouldn't - you should be able to get them from gstreamer.com or a Linux distribution). Just be careful with either FFMPEG or GStreamer that you're not picking up the GPL builds.

Gstreamer is a total nightmare to install and is flaky as hell. It hardly ever works properly and you can't "package it" with your game in any reasonable way. And if you do it is unlikely to work.

This solution is nice not because its ffmpeg, but because its just a file filter. We can easily use gstreamer or mplayer or whatever if it takes a some media file and spits out decoded frames and sound. The hard part of the code is done. The sync.

Media library's out there right now are total crap. They have an api to decode a frame and some sound... even working out the colourspace and then the format of this decoded frame is a PITA, you still don't have a solution to one of the bigger problems. Audio sync. By the time you have a binding and code that plays something... its a massive amount of work.

This is almost perfect for cut scenes and video "want to know more" stuff. I want to add skipping forward (think this is easy), pause start, restart(not really the player here), and stop (with fade perhaps). This is not a movie player.

The other thing that is fairly easy to do is have render to geometry. So i can put a TV, or movie screen in my game, not in front of the game.

Finally its simple. Here play this. Done.

I have gone through so many of the different bindings for java over the years. None of them really work properly or have all sorts of issues or require so much work just to play something even without sound. Or serious issues with performance. This puppy played 2k just fine, and with my changes its going to be able to play 1080p on pretty low end hardware.

But feel free to use all these other "easy just use this" libs out there for java movie playback.

I have no special talents. I am only passionately curious.--Albert Einstein

GStreamer used to be a bit flaky to use / install on Windows and OSX, but there's been a real push in this direction recently too - see http://www.gstreamer.com/. I still prefer the local packaging option for anything but Linux though.

All the extra things you're looking to do I'm already doing with GStreamer. For a simple use case I can see the benefit of Riven's code, but for doing more I don't see the point in rewriting what GStreamer already provides, but in Java!

I would agree, but custom formats don't solve anything really. Also i should be clear that this is not for user files, its for my prepackaged and encoded to be nice with my player files. I don't want to have a gp movie player. Just something that is easy to have prepacked and that will work.

Currently i will be encoding to theora and using ffmpeg, or a theora decoder. These can be statically linked binary's and avoid dependency hell.

last time i tried jgstreamer I just got a bunch of segfaults and was told its my system that is at fault. That is not the kind of support i want to provide.

I have no special talents. I am only passionately curious.--Albert Einstein

@nsigma You linked to thread that backs up my point with gstreamer. You can't do that for everyones install when i comes to release. That is where i have never had a good time with gstreamer. Sure you can get it too work. Eventually. But never generally.

I have no special talents. I am only passionately curious.--Albert Einstein

@nsigma You linked to thread that backs up my point with gstreamer. You can't do that for everyones install when i comes to release. That is where i have never had a good time with gstreamer. Sure you can get it too work. Eventually. But never generally.

Did you read to the end??? Cero's got this working fine (as he mentioned earlier in this thread). You can package the libs as part of your installer, and it just works! It's no different to Riven's solution in that regard.

@Cero, any chance you could post the code you're now using somewhere (my code is currently a little too tied to Praxis' architecture)? It would be good to get that code in a more generalized state, possibly using Riven's API as he suggested earlier, then we could do a serious comparison. I should have some free time next week I can look at it.

This puppy played 2k just fine, and with my changes its going to be able to play 1080p on pretty low end hardware

Just to get everybody informed on this one, delt0r is writing a YUV streamer, as opposed to a MJPEG streamer.

For those who don't know what YUV is: it's basically 'raw' encoded RGB - every pixel takes the same number of bytes. Most movies use YUV as their colorspace, so decoding to this format is always in the decoding process, hence this 'transcoding' doesn't take any performance.

This YUV data is piped to Java, which either decodes YUV to RGB on the CPU or on the GPU. In the end, there will be barely any overhead to the streaming of video, apart form the piping process, which would allow for playback of high resolution video on low end hardware.

Hi, appreciate more people! Σ ♥ = ¾Learn how to award medals... and work your way up the social rankings!

Most games don't need to play user provided video files. Would it simplify anything to use a custom format? Eg, Bink.

Why limit a developer provided video files to a specific format?

I can imagine not every developer would feel comfortable transcoding their video.

For the same reason your library limits a developer from being able to pause, resume, set volume, etc.

If the goal is simply to play videos, it doesn't matter what format the videos are in, as long as it is reasonably sized. Some portable, straight C code for playback would be great. There is a reason Bink is so successful. Transcoding videos hasn't hindered it. Of course, designing a video format and a decoder is not trivial. Possibly limiting to an existing simple, reasonably easy to decode format would be acceptable.

Cero's got this working fine (as he mentioned earlier in this thread). You can package the libs as part of your installer, and it just works! It's no different to Riven's solution in that regard.

@Cero, any chance you could post the code you're now using somewhere (my code is currently a little too tied to Praxis' architecture)? It would be good to get that code in a more generalized state, possibly using Riven's API as he suggested earlier, then we could do a serious comparison. I should have some free time next week I can look at it.

I'd like to emphasize that this should not be a competition (although that's not always a bad thing). At the moment I'd regard 'private code that works' not quite useful for the community, so I look forward to open sourcing Cero's solution.

Hi, appreciate more people! Σ ♥ = ¾Learn how to award medals... and work your way up the social rankings!

There is a reason Bink is so successful. Transcoding videos hasn't hindered it.

I always assumed that the reason Bink was so successful is that it created a standard - it could have been any codec, but by limiting yourself to one, there is only 1 codec to ship to the customer.

Yes, I know that 'contradicts' what I said earlier, and might undermine my 'YUNPM' project, but well, I didn't quite start this project because I need it myself, I just (think I) saw a need, and I came up with this simple solution, that is now being refined in the performance area.

If it won't be used, because all JGOers would use Bink, that's perfectly fine. In the meantime I enjoy tinkering with it and enjoy the process (releasing, feedback, updating, cooperation, etc).

Hi, appreciate more people! Σ ♥ = ¾Learn how to award medals... and work your way up the social rankings!

java-gaming.org is not responsible for the content posted by its members, including references to external websites,
and other references that may or may not have a relation with our primarily
gaming and game production oriented community.
inquiries and complaints can be sent via email to the info‑account of the
company managing the website of java‑gaming.org