31 Marzo 2022

Roland "Wave and SuperNATURAL Expansion" : compressed and 16-bit linear formats

On March 16th Roland presented the new Synth/Workstation Fantom06/07/08 which, due to the 256 MByte User Memory Flash, asked a whole series of questions on the real capacity of this memory that shares and can host the "Wave Expansion ”(EXZ001-EXZ015), the“ SuperNATURAL Expansion ”(EXSN01-03) and the User samples and multisamples that can be used within the Zen-Core synthesis.

The main doubt arises from the fact that the new Fantom0 are sold with the three expansions SuperNATURAL Acoustic Piano1 - SuperNATURAL Acoustic Piano2 and SuperNATURAL Electric Piano 1: these expansions, which among other things can also be canceled, occupy 120 MByte (46 % of the total capacity) leaving a free space equal to 136 MByte which is what you get when you buy the Synth.

What has created considerable perplexity is that in these 136 MBytes it is possible to load as many as 4 EXZ expansions, despite the fact that each of these is a file with a size between 96 and 128 MBytes. How is it possible to upload files larger than 400 MByte, in a flash space of 136 MByte?
Thanks to the possibility of canceling the 3 SuperNATURAL expansions, it is possible to completely free the 256 MByte of the Flash and then load up to 8 EXZ expansions, despite the overall capacity of the files (downloaded from RolandCloud with the RC Manager) is about 900 MByte, more than three times higher than the available Flash.
Another question to answer: how is it possible that the files of the three SuperNATURAL expansions, which occupy a total of 440 Mega, can only occupy 120 Mega when loaded into the flash??

The answer is very simple: from the files downloaded via RCM (Roland Cloud Manager) and loaded into Fantom0 via USB, only a small portion of the data present in the files is transferred to the Flash.
Same thing happens for other Roland synths such as RD-88, AX Edge and Jupiter X/XM.

To understand better, it is necessary to take a small step back in time and go back to the years 1988-1990 in which Roland presents the family of Modules / Synth U-110 / U-220 and U-20, D-70: it is the first step of Roland in the use of Multisamples which are stored in ROMs and Expansion Cards.

To save available space, Roland uses a compression technique called RS-PCM, most likely an acronym for "Reduced Size PCM". This coding, probably not particularly efficient, is immediately abandoned.
It is a first step towards the engineering of a new compression codec called FCE-DPCM which for the first time is used in the Flash of the JD-800 (it is 1991): all subsequent Roland synths, from the JV and XV family of the years 90, passing to the various GM/GS modules, up to the very recent Integra-7 (2012), FA-06/8 (2014) use this compression coding.

Roland has never disclosed the characteristics of this codec for obvious copyright reasons, but has given the possibility to users "geeks" to browse about this format thanks to the fact that it is used for example in the various expansions freely downloadable on axial.roland.com or on the SD card inside Integra-7.

To date, the only user who declared that he was able to decode this codec was the WebMaster of the dtech.lv site which among other things, to test its capabilities, made available, a few years ago, the precious content of the FlashROM of the samples of the JD800 in the original version (3 MByte compressed in the FCE-DPCM format) and in the corresponding clear format at 16 linear bit (size just under 6 MByte).

Regarding the quality of this codec, it is known that it has a variable compression ratio that reduces the size by a factor between 1: 2 and 1: 3 and that depends on the type of source samples (we will see it shortly): the author who decoded it describes it as highly optimized, so much so that it has a dynamics comparable to that of a linear 24-bit sample with small artifacts evident only in the fastest attacks.

In 2019 the Fantom arrives and for the first time the multisamples used for the external expansions are in the linear 16 bit format: the download of the Wave Expansion expansions (EXZ001-EXZ015) could always be done via http://axial.roland.com / as there was no Zenology PRO yet, which debuts the following year (May 12, 2020) as part of the Roland Cloud package.

These EXZ files, now unusable with the new Fantom firmware, have a reduced size compared to those that can be downloaded today through RCM as they contain only this version with samples encoded at 16 linear bit and therefore not compressed.
With the advent of Zenology, Roland adds two new features: the first is the insertion of license information (48 Bytes) within the Wave Expansion that make the EXZ files downloaded from Roland Cloud unique for a handful of bytes and associated with the account that created them; the second novelty is the inclusion in these files of the version of the multisamples compressed in the FCE-DPCM format.
The double format is also present in the “SuperNATURAL Expansion”..

Here is a shot showing, for example, the EXZ009 - EXZ Symphonique Strings expansion in the version of the Axial site (multisamples in 16 linear bits) and in the one downloadable with RCM. The compressed part has the same size as the one present in the old expansions for FA-06/08.

The presence in the EXZ files of multisamples in double format (compressed and 16 bit linear) and the insertion of the 48 bytes of license, guarantees Roland a whole series of advantages:

- loading of the versions compressed in Fantom0 (user flash 256 MByte), AX Edge, RD-88 and JupiterX / XM (they have free flash just over 64 MByte) and loading of the linear 16 bit versions in Fantom
- loading of the licensed versions (perpetual or subscription) in Zenology
- distinction, thanks to the 48 bytes of license, of the free versions for Fantom and Fantom0 and the paid ones for AX Edge, JupiterX / XM and RD-88

Regarding the three "SuperNATURAL Expansion" (EXSN01-03) the structure is identical, with the only difference that, at least for now, it is possible to use these expansions only with Fantom and Fantom0

Below are some shots showing the size of the two versions in the files, the empty space (a series of bytes equal to zero) and the compression ratio between the compressed version and the linear 16-bit version.

By analyzing the dimensions, the questions posed at the beginning are clearly answered.




A few brief observations.

The first is that the compressed multisamples present in the EXZ files all have a size equal to 32 MByte: this is consistent with the fact that they derive from the SRX expansion cards, all with a maximum capacity of 32 MB and compatible with the old series of Fantom, the XV family (5080, 5050, 3080, 2020) and JV (1080-2080).

The second observation is that the compression ratio of the FCE-DPCM codec, as seen in the table, is not fixed: this is typical of lossless codecs and codecs that introduce only small approximations in the compression phase.
The compression that can be obtained is strictly linked to the characteristic of the source file.

The last observation concerns the sound quality: how much does a Fantom0 that reads multisamples compressed with the FCE-DPCM codec sound “worse” than the Fantom that reads 16-bit linear files (obviously we exclude the impact of converters)?
I would not assume that for the 15 EXZ, there is a deterioration in quality because it has to be shown that Roland with Fantom has restarted from the original uncompressed samples (material from the 90s) and inserted them into the EXZ directly: it could be that the choice to use linear 16 bit encoding is due to the Fantom architecture which is very complex as can be seen from the following diagram and therefore obliges the BMC to work with necessarily uncompressed samples.

In fact, in Fantom, all the usable samples (internal, expansions and user) are stored in a 4 GByte eMMC memory and when switched on they are entirely transferred to RAM (also 4 Gbyte): while the sounds are synthesized , the necessary data is distributed to the small RAM (32 MB) associated with each of the 4 BMCs, through 2 FPGAs Altera Cyclone V.
AX Edge, JupiterX and presumably Fantom0 instead have a more classic architecture where the samples always remain in the Flash and are loaded, when necessary, in the small 32 MByte memory that accompanies each BMC.
Therefore, it cannot be excluded that Roland, for the conversions of the old SRX cards in EXZ, in Fantom has simply converted the compressed files into 16 linear bits, obviously without being able to improve their quality, "forced" only by HW architecture reasons.

Regarding the most recent sounds (see for example the SN AcPiano2 and the first 4 EXZ expansions that do not derive from the old SRX cards) certainly the uncompressed nature of the samples guarantees in Fantom a slightly better quality than that of Fantom0 which is forced to use the compressed version. The excellent quality of the FCE-DPCM codec is, however, a guarantee of an almost imperceptible difference.


I conclude with a video I made a few months ago of a "Teardown" of my Roland AX-Edge with images and hardware details: regarding the part relating to the sound generation, the mainboard audio contains a single Roland BMC SoC, 48 MByte of RAM and 256 of flash that contains up to 2 EXZ expansions and about 150 MByte of firmware and ZenCore samples: in addition there is a second smaller and slower Flash (48 MByte) probably used for storing tones and performances.