Quality Test - H.264/AVC Game Recording with the x264 AVC Codec, with the goal of Quality (with "2K" HD Video Samples from Four Games)

In an earlier post about game recording with H.264/AVC (MPEG-4 Part 10) here, I went somewhat in-depth about concepts and considerations for the various settings and why you might want to use them or leave them as they are. With plans to make a "short version" of why you might want to use some settings over others (and setting recommendations geared more towards Speed of recording, reducing lag/choppiness) in the near future, for now, enjoy this short sampling of some game recordings done with H.264/AVC - this time with the goal of increased Quality in mind as well - showcased using four different games and presented in 2K Resolution (2048x1152p):

Clicking the Gear icon in YouTube's player and choosing 'Original'/1440p, while still Youtube-recompressed, is the closest to the original upload in viewing quality (You may need to click on the YT logo and watch it at YT to enable the Quality Setting)

Recorded games:

Battlefield 3 (Grand Bazaar), Planetside 2 (48+ Players Per Side in a BioLab territory, Night), Hitman: Absolution ('Run For Your Life, Shangri-La'), Grand Theft Auto 4 (Broker area, Night)

These games were selected because I felt they created a good 'obstacle course' for this codec Test, with light and dark areas, large panning/movement areas and hard edges to deal with, either from on-screen text or finely detailed textures [all Texture Filtering was turned Off in the Videocard Control Panel settings] (they were also all relatively demanding games, showing then, the low performance hit of this codec, with these settings)

Recording codec:

H.264/AVC configured with the x264 Video For Windows Unofficial (Black Logo) interface (the XiWave GNU GPL MPEG-4 Codec)

Recording settings:

CRF18, NoPartitions, Fast P-Skip, 1 Reference Frame, Diamond ME, ME Range 4, Subpixel ME 1, GOP 1 (for editing with Vegas/Premiere), NoWeightedP-Frames, No B Frames, Deblocking Filter (Strength 1), Non-Interlaced, No CABAC, DCT Decimation, No Trellis, Deadzone (11,21), Flat Matrix, MaxBitrate 50000k (Buffer 5000k), Threads 4, all other settings left as Default/Off

Recording compression:

Originally recorded @ 50000kbps, H.264/AVC format @ HD Resolution (1920x1080p)
Compressed @ 25000kbps for smaller upload to YouTube, WMV format @ 2k Resolution (2048x1152p), in attempt to keep detail/compensate for YT recompression

[The 'analog pause' effect wasn't added to accentuate any specific action going on at the time, it was more to showcase the detail maintained at that time/frame, when recording with the h.264/AVC codec] 

At a max bitrate of 50000k (a high quality Blu-Ray movie bitrate), the original recorded output takes up about 375MB per minute of recording. For an hour of gameplay, that's less than 25GB (including uncompressed audio (uncompressed to be more compatible with video editing apps and use less CPU resources while recording)). Maintaining good quality - although quality is relative, mind you - that is still about one-tenth the filesize of a FRAPS1 codec recording or a YUV or Lagarith codec recording. Chalk it all up to a powerful codec that has the ability to not only compress more where it can (without over-compression in dark/flat or low-motion areas), but also compensates for its' own compression via Deblocking and other built-in techniques.

Doing a quick test without the self-compensation capabilities found within the codec (turning off Deblocking, CABAC, etc), the output has macroblocking and other artifacts occurring, as can be seen in this screenshot:

An example of the internal filtering/compression-compensation found within the h.264/AVC codec, two frames taken directly from two h264/avc recordings, done one after another in the same area in the same online game
(Rift, recorded @ 1080p).
Click to see Full Size (cropped and zoomed source)

The two frames above are from videos shot close together in time, at the same bitrate, resolution, everything... The only difference between the two is that the left side has all filtering/compensation for compression that the codec can perform 'OFF' and the right side has CABAC, Deblocking and other compression/correction techniques 'ON' (Deblocking within the codec was set at only +2+0, as higher Deblocking settings have the effect of excessive smoothing.. [but if you don't mind that or you like the look of it, you can turn up the Deblocking to a maximum setting of +6+6, with a small hit to recording performance]).

These powerful compensators for compression come at a tradeoff however, as the more that you turn on, the longer the codec will take to scrutinize the frames and the slower the compression/output will be. This results in 'lag' both in the game and in the output video, as the resources of the system are directed more towards analysis and compression of the recording, where doing the calculations on the frames, then writing them to disk, can cause the recording to fall behind what is actually occurring on the screen. Also, turning up too many compensators/filters within the codec can result in a 'washed-out' (or at least excessively-smoothed out) video, meaning higher loss of detail.

So, are you stuck with only either crappy looking video or too-smoothed-out video output? Not at all. There are two main directions you can go with your h264/avc game recording (which one you decide on is limited somewhat by your system capabilities):

• On the one hand, you can have small filesizes, allowing you to record longer and more video. You can turn up the analysis and compensation techniques within the codec (Deblocking, Motion Estimation, CABAC calculations, Partition Analysis, etc) - as much as your system can handle, but not too much or you risk excessive smoothing, lag and other effects - and still end up with good quality game recordings.
  
• On the other hand, you will have larger filesizes, but if your system cannot handle the extra analysis and calculations of the codec, you can turn off these compression/compensation techniques and also gain more speed/performance [less 'lag' during gameplay and in the video output - this may also result in more detail being kept from the original source]. All you have to do is allow more Bitrate (setting a lower CQP or lower CRF, if you are using those) to more accurately represent what is occurring on-the-screen/in-the-frames, so that you don't get compression artifacts (as in the left portion of the screenshot above) - and still end up with good quality game recordings.

What settings to use, then? I will present the settings I settled on, after much testing; but again, these were done on my system, with the limitations that includes for me. You may have a more powerful or lesser so system, but all it will take is a few tests, changing a few settings each time, to see what kind of quality you can arrive at, with the balance/tradeoff of performance hit you are willing to tolerate.

Although JPG-compressed for posting, it can still be seen in this frame extracted directly from the H.264/AVC game recording (Hitman: Absolution @ 1080p), that decent quality can be maintained, despite the ability to record in smaller filesizes, with this codec and these suggested settings for the x264 interface.
Click to see Full size

There was only a light performance hit for my system with these upcoming settings, usually only a couple frames per second (averaging about 5fps loss, depending on the game/area being played); but it may depend on the 'overhead space' that you have running a game already. What I mean is, if a game is already chugging along for you, you may not be able to use or turn many things on with this codec (it can be much more demanding, as the different processing options get turned on). Along that same line of thought however, if you are able to run a game you want to record smoothly already, you should have no problem trying out this codec (esp. with my suggested settings). Since that was the case with many of the games I was playing with at the time [although games like GTAIV are always choppy haha], I will list the basic system specs that were used during the time of this Test:

AMD FX-6100 CPU @ 4.0GHz
AMD/ATi Radeon HD 6870 GPU (1GB VRAM) @ 950MHz
16GB Patriot "Gamer" RAM
all on a Gigabyte 990FXA Chipset Mainboard
running Windows7 64-bit
recording to two SATA III harddrives set up in RAID 0 (~233MB/s throughput, according to Dxtory)

I also managed to perform these couple of quick tests, as a comparison of the performance hit to be expected when recording with this codec and these settings on some other systems (CPU/GPU combinations listed):

Unigine Heaven Benchmark
6-core CPU / HD 6870 GPU ~1-3fps
4-core CPU / GTX 560 Ti GPU ~3-5fps
2-core CPU / GTS 250 GPU ~5fps

Rift (Online MMORPG)
6-core CPU / HD 6870 GPU ~1-3fps
4-core CPU / GTX 560 Ti GPU ~ 5fps
2-core CPU / GTS 250 GPU ~5fps

Not bad, in my opinion. These settings seem to work well, no matter what combination of hardware there was. Hopefully this will be the same for you, dear reader.


So then, just below are the settings I normally use when recording with the H.264/AVC codec. I might change things a bit for certain games that look vastly different, such as, if I wanted to keep the grain in the Left4Dead series of games or if I was recording a solid-coloured/static-area looking game like Minecraft or Web Browser games. For the most part however I use these settings, both for maintaining good quality and keeping disk space usage relatively low (at least, far lower than using the FRAPS1 codec, Lagarith, or a YUV codec).

[Note that I have chosen to use a GOP (Group of Pictures (Frames) (called "keyint" in the bottom example)) of "1". This is for editing compatibility with Vegas/Premiere and is optional if you are not using these video editing applications. Feel free to change the "1" in this setting to the Codec Default of "250", or as desired. For more information on this, especially if you plan to edit the recordings with Sony's Vegas or Adobe's Premiere lines of products, see this post earlier, here]

When using the 'Unofficial' (Black Logo) x264 Video For Windows interface to configure settings:

When using the 'Official' (Red Logo) x264 Video For Windows interface to configure settings:

For the official/redlogo interface, choosing the Ultrafast Preset configures many settings that are the same as the above prior three screen captures, seen using the unofficial/blacklogo interface. The command line area (box at the bottom) where things are typed in, change the remainder of the settings so that they end up the same as the other interface settings in the prior three screen captures above.

As you can see, these settings can be configured using the x264 interface for the codec, so no matter what game recording program you use* - be it Dxtory, Bandicam, MSI's Afterburner, etc - you will be able to choose these settings, adjusting for your own tastes as you prefer (or adjusting to be within your hardware limitations). 

[*The game recording application must be able to use 'external/third-party codecs', i.e. other codecs not included with the program itself, that you have installed on your system, in order to utilize the x264 interface for the h264/avc codec]

For more detailed coverage of the settings used when recording with h.264/AVC [in an article that focuses more on speed of recording, reducing 'lag'/choppiness], where to download the codec/interface and how to start using it in Dxtory, Bandicam and MSI's Afterburner, see this earlier post here at the blog:
http://gametipsandmore.blogspot.ca/2013/05/game-recording-with-mpeg-4-using.html

Please note dear reader, that I am not saying "This codec is the best one to record with" or "use this one only".

I am merely showing that it is possible, or how to tweak it for quality or file size, as to your own personal preference.

There are many codecs out there to choose from when game recording and although some are more apt for certain types of games than others, overall it is your own choice to use whichever one you prefer. Do some short tests and see which one works best for you and your current system - and try to have fun with it

Have Fun Recording with H.264/AVC and See You In The Games!