ReSID filter distortion simulation

This is a fork of sidplay2 and associated libraries, with primary purpose to improve emulation of the C64 system and the SID chips themselves. We have also designed a SID chip measurement program for C64.

Summary of modifications

SidUtils

• Sidutil's external filter definitions get 3 new parameters to pass to the distortion model.
• Type 3 and Type 4 filters are defined.
• Type 1/2 filters are removed.

ReSID

• ResSID's 6581/8580 filters are tweaked for distortion and phase.
• Debian version bugfix: filter terms were calculated in wrong order, resulting in boosted highpass output and phase distortion.
• SounDemoN provided a fix for noise register control with the test bit, which his song Bojojoing uses. When test bit is on, the noise bits eventually all become set due to some kind of analog effect. This is emulated to occur simultaneously after 2 seconds.
• SounDemoN provided a fix for combined noise waveform lockup, which is used by some emulator test sids such as nata's emutestb.sid. Combined with the test bit effect, it allows further control of the noise waveform.
• Waveform 0 outputs the previous oscillator value. In the real chip, the analog fade destroys the bits eventually, but this is not emulated.
• I added waveshaping of the waveform generators. It looks like there's a lot to improve on the waveform DACs, but the principal effect missing was some kind of saturation of the wave and envelope output. It is now roughly emulated by similar kinked DAC approach as the filter.
• I added hardclipping to the 6581 mixer strip. It would appear that there is strong saturation effects at the output amp, near the extreme range. Of course, not on all chips, so it should properly be a tunable.
• I removed the "fast" modes. Everything runs at full accuracy. This limits fast forward speeds to 4x-16x on typical today's hardware.
• Combined waveforms are calculated, not tabularized.

LibSidPlay

• Now uses ReSID's output generation code without aliasing.
• Libsidplay used to simulate 2 SID chips even if the song only used one. This wasted a lot of CPU on higher quality modes.
• 8-bit soundcard output is gone; only 16 bits are available
• Mixer's Iisibiisi is fixed. 6510 emulation had a NMI timing bug. Also, another associated CIA timer reprogramming delay bugfix from Simon White was included.
• Fixes to RTS/RTI/BRK instructions that share the hardware circuits with JSR and JMP, and thus read next 2 bytes after the opcode. Fix to branch instruction timings.

resid-builder

• sampling() method is called automatically by libsidplay when the song's clock frequencies become known. Clients can no longer call this method directly. (Interface change would be required in any case because it is necessary to know the cpu clock rate to calculate the samplerate.)

sidplay2

• Added Pulseaudio support and ALSA from upstream. Pulseaudio is preferred, and OSS is chosen as last resort. The Pulseaudio's ALSA plugin seems to work rather poorly, with tendency to stall during underruns, and upstream ALSA code in fact failed for me with "Socket operation on non-socket" error message, which I was unable to debug.
• New debug key commands: 1, 2, 3 to toggle voices, f to toggle filter. Numpads keys for navigation don't work any more.

Download

Use the https://sourceforge.net/projects/sidplay-residfp/ to obtain sources.

The sidplay2.ini file contains all the chips I have modelled. I also have them as separate files, but these are not versioned and might not work! All libsidplay2 clients need to be recompiled. To quickly get to testing, install my sidplay.

Some sample songs with the patched ReSID

The reference chip appears to have a dead noise generator on channel 2. That's why some songs have missing sounds. The missing songs are due to SIDs that do not play on genuine C64 using the MMCReplay sidplayer. I have since solved the problem with psid64, but have not yet sampled the missing songs.

Some songs have been rendered with different filter setting in addition to the R4AR setting. These reflect my belief of how the songs should sound like. Remember, 6581s are different.

Distortion theory and practice

The programmer sets an 11-bit register called CF to one of the 2048 values. These values are converted to a voltage inside the SID chip which drives the active amplifiers that implement the filtering circuit. This conversion is unique to each chip, but there seems to be a lot of clustering in the results, possibly due to manufacturing plant or materials used to make the chips. Chip revision does not seem to strongly correlate with the curve position, although there are not enough chip characterized yet to say anything for sure.

Caution. This is written by an electronics noob, so if it sounds like twitterings of a madman, please help me write this section properly. Below is the circuit diagram for the filter. This does not have all the outputs marked, but you can imagine that there are wires connected to the outputs of the three op-amps on the top row. The leftmost, highpass output, is actually shown.

The top three opamp outputs are Vhp, Vbp and Vlp. Two of the resistors marked as R2 are driven by CF, and the R3 is driven by resonance on the 6581.

From this schematic, the following digital equivalent is derived:

        Vlp -= w0 * Vbp
        Vbp -= w0 * Vhp
        Vhp = Vbp / Q - Vlp - Vi

According to Kevtris, the distortion arises because the two variable resistors R2 between the op-amps in the top row are poorly implemented. A R-2R ladder DAC from the FC register, is connected to two FETs, which in reality are the variable resistors. In truth the w0 term seems to be physically implemented by a regular resistor with a FET connected in parallel. The fixed part is the long squiggly resistor inside the FET circuit on 6581 die shots.

The FET is connected up using source-biasing circuitry that makes it a linear voltage-controlled resistor over the ohmic operation region of the MOSFET. (Lord Nightmare suspected that the fact the source and drain are not connected to a fixed potential causes weakly exponential behaviour instead of strictly linear one.)

This image is a small detail from ZrX-oMs's full coloured SID die. Shown is the voltage-controlled resistor (VCR) believed to be responsible of the SID distortion. The +12V line biases the green polysilicon layers which makes the red n-well layer act as resistor. The longer resistor on the top is modelled as baseresistance in type3 equation. The FC and source mix together through the two resistors in the lower part, and the resulting potential is connected to the FET gate, which completes the VCR.

There are some unused pieces of the blue metallization layer over and near the large ~2M resistor. They are assumed to be leftovers from Yannes's earlier design.

Simulation proceeds with this kind of procedure:

        w0_hp = distortion(Vhp)
        w0_bp = distortion(Vbp)
        Vlp -= w0_hp * Vbp
        Vbp -= w0_bp * Vhp
        Vhp = Vbp / Q - Vlp - Vi

where the distortion() function is based on type 3 equation of two parallel resistors:

	# parameters:
	#
	# baseresistance, minimumfetresistance, offset, steepness = type3 params
	# kinked_dac() from type3.
	#
	# value ranges:
	#
	# kinked_dac(fc) between 0 .. 2048.
	# rate matches the input and fc voltages together. In ReSID,
	# this value is 1/128, and one voice has full range of 4095 * 255 * sqrt(2)/2.

	def distortion(input):
		dist = (input * rate + kinked_dac(fc)) / 2 - 1024;
		if dist < 0:
			dist = 0;
		# dist == 0 for no distortion, pure type 3.
		fetresistance = offset / exp((kinked_dac(fc) + dist) * steepness)
		dynamic_resistance = minimumresistance + fetresistance;
		resistance = (baseresistance * dynamic_resistance) / (baseresistance + dynamic_resistance)
		return 1 / (clock_frequency * capacitor_value * resistance)

The distortion function above injects the dist param once the voltage at gate exceeds a threshold. This threshold seems to be experimentally about 1024, or half of the full range of the fc adjustment. It is likely this threshold relates to the ohmic and saturation regions of the MOSFET. Details are still a bit sketchy, though.

Normally, FET entering saturation region means that the source-drain current becomes almost independent of the source-drain voltage (as long as the saturation condition holds). However, in the VCR configuration the source is also connected to the gate, which provides a quadratic increase in current with response to drain voltage!

One way to look at it is to think that once the level increases past a point, the FET conductance increases more rapidly, as if a larger value was stored in the FC register. This increase is also adjusted by the level of the voltage supplied by the FC control: experimentally, the higher value in the FC control, the easier the distortion becomes to trigger.

Recent changes

7.6.2009

V36 is a simplification from V35, because the previous version was somewhat odd/artifacty. In order to continue work from sounder base, I have simplified the code to as far as I have been able to. I removed most of the state variable mixup code and gave up with certain tweaks I was never sure about.

23.04.2009

V35 is out. This is incremental improvement from V34, which I did not mention much about. All the work has been in subtle finetuning of the combined waves and filter. I think the combined waves should be really alright now, and I fixed a bug with Laxity's String that was sounding pretty bad with V34. I deleted a ton of likely-useless audio drivers and changed the output format to floating point. Build files are for MSVC2008 now only.

07.03.2009

I released V33. People who want to compile this for Windows are likely to see some problems because couple of duplicated header files have been moved. The autotools build symlinks them. Besides build changes, I have fitted the combined waveforms using better algorithm and finetuned type3 curves with a fitter program I wrote. There is also some changes in the filter state variable mixup code.

22.02.2009

I released V32. I fitted the combined waveforms with aid of a C# program, and fixed numerous bugs left in the original release of V31.

17.02.2009

I have been lazy with changelogs. There are no entires for V29 and V30.

• some small filter tweaks.
• 6581 clipping retuned.
• voice/wave offsets restored back to reSID's original values.
• some correctness fixes such as avoiding use of uninitialized values.
• combined waveforms are now calculated at runtime. The new waves sound different, especially on Mixer's Dawn. The PT, PS and PST waveforms are probably better now than before. I modelled the waveforms after kevtriis's chip C, which is one of the more extreme 6581s, with relatively loud combined waveforms. I'm likely to change the combined waveform sound slightly, to a more edgy/dryer sound on both 6581 and 8580.
• Leandro Nini provided me with fixes for the build system, now "autoreconf -ivf" should correctly generate all the autotools junk to build sidplay-libs.
• untested build fixes for hardsid-win.cpp.

This release is slightly more a technology preview sort of thing than a full production quality code. There are likely some small issues here and there, both with the combined waveforms and the filter. Nothing too drastic, though, and most music is perfectly listenable, and some probably better than it was before. I'm interested in hearing about regressions, though.

17.12.2008

V28 released. V27 was released 2 weeks ago, but I forgot to update this page. Sorry. V28 supercedes it anyway, and is loaded with goodies. I fixed Fanta In Space (Fanta used instruction prefetch side-effect in 6502 RTI to clear NMIs, and branch instruction timings were also invalid in the CPU emu).. The same fix apparently fixed Human Code Machine's MDG part from Sid Vicious.

I toned down 6581 resonance because I found out that it was responsible for the nasty broken sounds in Wizball when using trurl6581r3_0486s filter. Also Hard Track was a bit too "edgy". On the other hand, 8580 clearly has more resonance than 6581, so I kept 50 % stronger resonance for 8580 tracks.

Otherwise I've done some small tweaks to 6581 filter but nothing worth mentioning.

15.11.2008

V26 released Still some changes in filter state variable mixing. bp is now about 20 % louder than before. Mixing is performed in the level-adjusted form, which seems to help immensely against most music. The overall effect is more convincing distortion sound, but also slightly muffled. Snapping-related problems reported by Nata and nojoopa should be greatly reduced.

31.10.2008

V25 released Main change is a slightly different filter state variable mixing, some bugfixes for JT's Star Ball, and even more resonance. I also battled several hours with autotools (again!) to get this to compile on Intrepid.

9.10.2008

V24 released. I added a new effect based on the idea that the highpass filter VCR element adds a negative offset to the bandpass filter. I decided to give up with the strong lowpass filtering approach on V23 and just use a +3 dB bass boost. I think no state variable approach for producing the output level differences will work, or is even correct.

I'm very happy with the sound of this version for the most part.

25.9.2008

V23 released. VICE & work has kept be rather busy lately.

That being said, I've went for too long without a release. Main changes are:

• Added another 10 % to resonance control range
• Changed distortion algorithm with powerful lowpass-filtering style effect for bp/hp outputs
• Increased filtered sound level slightly to compensate for loss of treble level due to filtering
• Minor code syncs between VICE resid and mine.

30.8.2008

Regenerated the build system, removed files that were in my SVN but which in fact could be generated by autotools scripts. Sources now distributed with "make distdir". No more patching, for better or worse. Numerous MSVC compile/warn fixes. Optimized voice output slightly. Re-released V22 in the new format with small, unimportant tweaks.

29.8.2008

Optimized resampling to be about 2x faster through using aliasing to increase passband width and hence reduce FIR length. Fixed some valgrind warnings. Small tweak to distortion algorithm. Strenghten SSE requirement through -mfpmath=sse to ensure that SSE registers are used for filter and everything else. Added fallback path to systems where denormals can't just be disabled, such as Java. The old code for ST_lockup proved unnecessary, so it is now removed.

With this patch, I'm reducing focus on sidplay2 and concentrating on VICE. It is a faster C64 emulator, and the VSID mode could be tweaked to complete sidplay2 replacement. The work now lives in a branch, and will be available as new engine, the ReSID-FP (FP stands for floating point).

21.8.2008

V21 released. A few weeks without release -- the primary reason was the Psykoz2008 party last weekend, that precluded me from working on the emu properly.

V21 contains mostly infrastructure work. I removed a lot of code associated with the lower quality modes of ReSID, and only left the 1 MHz clock routines. This limits fast forward speed, as the full accuracy mode is rapidly outmatching the processor. Analysis shows that the filter code is a major culprit, but the digital side emulation is not completely cheap, either. I already added some small hacks to optimize it, and I now think about reinjecting pieces of those clocking modes that run the analog parts with reduced accuracy.

I also fixed couple of bugs in V20. The nonlinearity setting appeared to get lost, and kenchis discovered further bugs in it. Oh well.

I defined type4 filters based on equation of straight line. These are characterized by parameters k and b, and the equation is freq = k * fc + b. Strictly speaking, straight line is not 100 % accurate, and a 2nd degree component would help, but based on measurements against two of Trurl's 8580s, the maximum error introduced using linear approximation is mere 3 %, and that is already comparable to component tolerances themselves.

Also don't forget to listen to AMJ's Assembly '93 compotune.

10.8.2008

I ripped off Type1 definitions from the filter, and defined 8580 type4 filters as the equation of line, frequency = k * fc + b, where typical values for b and k are 20 and 6.

This allowed me to remove the spline.h PointPlotter class, and required me to rebuild the Makefiles. Incredibly, I was able to regenerate them using the autobreak toolsuite, and the only thing that went wrong was that libtool got option --tag=CXX that was not actually supported by the generated libtool. I have no clue why this happened, and believe that autotools is one huge pile of condensed WTF. I patched the libtool by hand to just ignore that option, and it builds again!

Later on, I figured it out properly. The ltmain.sh already in the project caused the libtool to build incorrectly. After I fixed that, I got the following awesome error: "relocation R_X86_64_32S against `a local symbol' can not be used when making a shared object". The error suggested adding -fPIC into the build options, which I did through configure.in. Well, it "works" again but my confidence in C++/Autobreak stuff is weaker than ever.

I removed the "fast" modes from ReSID. It now uses more CPU as everything is always rendered in full accuracy, but I can still fast forward at 4x speed, and 8x fast forward stutters much like it did before as CPU runs out, so the change was not much to the worse, and several classes are, like, half smaller now.

10.8.2008

Released V20. I changed the state variable mixup code from V19. I suspect the resistance to FC control is much less than the resistance between bp-hp, thus I added exponential decay that biases bp and hp towards zero.

I also added a bit of hardclipping in the output that seems to be required to get Fred Gray's Break Thru. This might not occur so strongly on all chips, so it should probably be added as a proper tunable. For now, I just settled for a slight effect to collect feedback.

2.8.2008

ZrX-oMs alerted me about Wizball's sound, and Grue criticized the sound of Laxity's Alibi. I heard nasty distortion artifact on Wizball with Galway filter, so I changed V19 a bit. I have rereleased it, and described the change in this posting.

1.8.2008

Released V19. This is a slightly courageous release, with many things that could potentially go wrong. But it sounds really good, I think. I have made major strides in Miami Vice, Wizball, SidRiders and SYS4096.

31.7.2008

Added every trick I have ever tried into the distortion model. These are: lowpass level boost; small resistors between lp-bp and bp-hp outputs; mixback through the output strip. I also changed the distortion algorithm, now using a best-of-breed expensive exponential with the cf-adjusted threshold, and breakup point set at the end of CF control range. The result: very impressive sound.

30.7.2008

Working towards V19. I added output strip mixing back into the filter. I think it occurs after all, only the effect is very faint (currently modelled around 1/6th of the effect of the direct hp-bp mixing).

25.7.2008

V18 consists only of rework, minor tuning and cleanup of the work that went into V17. I heard that V17's 8580 side was broken. I'm sorry about that. What likely happened was that I released the version with some uncommitted changes on my working box, and the scripts that I have written only publish the committed changes. Since it "works for me", I never noticed. This version, anyway, should work.

The largest difference to V17 is that I halved the resistor value between bp-hp. This seems to improve a lot of songs, so it's likely closer to correct now.

I also tweaked the voice DC offsets in the 6581 side, based on some comparisons against samplings that I have here.

15.7.2008

Another week, another distortion preview. In other words, V17 has been released.

This release is a bit slower than usual. The culprit is the changes in voice.h where a new kinked-dac style calculation is used to simulate the nonlinear shape of the waveform outputs. The cost of the new calculation varies by song as I was lazy and only cached the result of previous calculation, so speed depends on the pitch of the waveforms and the precise waveform chosen.

exp() is back, but this time it is implemented as an integer calculation according to Schraudolph's paper "A Fast, Compact Approximation of the Exponential Function". Using near-proper exp() simplified the distortion algorithm. I think it all sounds much better now.

A new effect where I mix bp-hp and lp-bp together by their difference to simulate a small resistor between them appears to improve SidRiders and several other songs, largerly eliminating some types of hard distortion sounds that do not occur on the chip. It also helped Mechanicus. I am trying to understand where this term comes from...

9.7.2008

I have released V16. I was able to remove the offset calculations, and I now have somewhat less arbitrary values for the distortion tunables. They are now related to the relative signal levels in the simulation. I'm still sorting out the particulars of the values I ended up with ("why 128 instead of 256"), but V16 is slowly making sense, and I hope I'm not saying just because I might have lost my mind. I exposed two new tunables in the process, to hasten experimentation with them.

30.6.2008

Started V16 with yet another distortion algorithm. Seems promising. Improves Ghouls'n'Ghosts using the R3 4885 filter and Hard Track using R3 4485, and Fist2 on R4AR 3789.

The main thing with this version is changing the level offset calculations for the output.

27.6.2008

I am an idiot. I finally went and simulated the filter from a small commandline program, to understand Lord Nightmare's new sampling that proved conclusively and beyond doubt that the bandpass phase of the filter is inverted. Here's where I was mistaken: it did not occur to me that the stock ReSID procedure already had the bandpass inverted. Shit.

So I wasted a lot of time some half a year ago trying to invert the bandpass phase and did not understand why that did not work, even when I realized from the filter diagram that it must be inverted. Stupid, stupid, stupid!

I changed the bandpass phase for 8580 emu, which afaik is not made of inverters, and hope that this improves the 8580 sound further.

26.6.2008

I released V15. I have tuned voice nonlinearity further, and changed the way voice offsets are calculated for 6581 and 8580 digi sounds. I hope that what I got is approximately correct, although 6581 digis may now be too quiet. That is matter of changing the voice_DC value.

I removed lowpass filtering on the distortion term, being able to replicate its effect through adjusting the distortion threshold dynamically based on the offset that arises from the FC DAC. This change improved SidRiders, but damaged Hard Track slightly. I think I need to work more on the distortion function next. (Counter-intuitively, the distortion has to be turned pretty high or Hard Track sounds even worse.)

10.6.2008

I released V14. I have removed the asymmetric filtering, reduced voice nonlinearity, and applied symmetric and rather strong lowpass filtering on the distortion term. I was also able to remove the parabolic distortion term in favour of purely linear one. I think this is what V13 probably should have been.

Additionally, I have pushed floating point arithmetics through the rest of the output code, restored the ReSID original resampling constants to their full 96 dB SNR & 10 % stopband glory. I found some unintentional double constants in the fast paths, which caused unnecessary runtime casts, and implemented the FIR convolutions with SSE. (Of course, I haven't really validated that it all still works correctly, but it sounds fine...)

7.6.2008

I released V13. The only new tweak is reintroduction of (non-tunable) distortion filtering, which makes the distortion begin quickly but fade off slowly. This has consequences largerly to music that uses loud noise-based hits like Fist II cave or Jeff's Hard Track. On V12 the distortion had no memory beyond the state variables themselves, and while it worked fairly well, I do not think it will ever properly reproduce the way noise is distorted.

This asymmetry suggests that the distortion has some NMOS-related reason, which characteristically is much better at pulling signal down than up. One is naturally lead to suspect the op-amps: perhaps the excess current observed is due to imperfectly cancelled input signal in the feedback circuit, and the mildly exponentially behaving FETs then make matters much worse.

I started V14 to address some criticism I received. I have basically doubled the recovery speed from distortion and made distortion occur earlier to compensate for it.

30.5.2008

I released V12. Release highlights, in no particular order:

I have added waveshaping of the envelope-modulated voicegen to improve Cane's songs. I might have overdone it a little, but I find the effect pleasing to listen to.

I removed the DistortionHighpass and -Lowpass variables. There is no longer any kind of frequency response based filtering in the distortion code, as the above fix seemed to finally fix a number of songs and made this tweak that must, somehow, have compensated for it, irrelevant. There might be some small improvements attainable from very slight lowpass filtering, something to the tune of 10 kHz lp filter, though. I suspect this relates somehow to the way I observe some kind of unexplained general low-pass effect on the highpass output. This effect seems to reach -3 dB point somewhere in the 10 kHz region, too. Maybe it is actually MOSFET gate capacitance, maybe it's something else.

A new tunable called DistortionPoint was introduced, and it is used to control the minimum amplitude required to begin distortion, to allow for quiet nondistorted sounds.

No expf() in the hot path (or approximations based on linear interpolation such as scalbnf()). Removal of this very expensive part should help with CPU usage.

22.5.2008

I have spent a few days trying to model the combined waveforms. Or rather, one combined waveform. I started with PS waveform. The model is simply as follows: the bits pull themselves with fixed strength to 1-state. For each 1-bit, I calculate a pull-down effect resulting from the surrounding 0-bits. The function is currently sum over 0-bit-specific weighing factor divided by distance to the 0-bit. If the pull-down effect is strong enough, the bit falls to zero.

This approach yields some 14-dimensional problem (12 weighing factors and couple of tunables), and I optimize it using Monte Carlo method. The algorithm is written in C# because optimizing in Python seemed rather slow.

At the moment, I'm still missing something because of the 32768 bits involved, I mispredict about 1600, based on the reference samplings. This is still too many, so I need to come up with improvements in the function. In sample terms, of the 4096 8-bit samples the sum of absolute difference between model and reality is 22000, meaning that on average there is an error of 4.5 8-bit sample levels comapred to the reference signal.

19.5.2008

New distortion algorithm. For the first time, I can hear Wizball approximately right at the same time that Fist II's Cave plays approximately right. I also got rid of the very slow exp() in the tight loop, which helps a bundle with CPU usage, down from 35 % to 25 % or so.

14.5.2008

Found a bug in my fc curve analysis program. Criminy. I have previously observed from my samplings that filter outputs have level difference such that the LP is supposed to be 3 dB louder than BP or HP. Well, this won't be true for 8580, so I had to redo my 8580 analysis. In the new analysis, one of Trurl's 8580s replicates ReSID's curve. The other is still off. I'll now redo all the 6581 chips and check the output levels.

Update. It looks like highpass output lowpassing. On my 6581R4AR the effect is worth about 3 dB. It's interesting that on some other chips the effect was much slighter. I will really need to check this for all 6581s. *sigh*

12.5.2008

I decided to release V11. I believe I let this code stand for a longer while, as this should really be pretty accurate distortion simulation. Further improvements should come from refinement of the distortion params and some tweaks to the distortion function. I should figure out what causes the level differences on the bp/hp outputs compared to lp. Understanding this part might improve the simulation quality significantly.

I also plan to remove the type1 definitions in V12, and define type4 filters for 8580, which would appear to be well modelled by a second-degree parabolic approximation (the 2nd degree term is almost 0, too).

11.5.2008

New distortion algorithm. I decided to revert the bandpass flip. Several songs show evidence that hp, bp and lp are all in the same phase. Funny, this used to help, but now that the distortion function is finally almost correct this works better again.

10.5.2008

Started V11.

Spent today analysing chips. Two 8580s, two R4ARs and one R2 from Trurl_Ext, two R3s from Lord Nightmare, two R2s from me.

Reduced distortion-patch sound level by 6 dB to avoid most of the hardclipping that sometimes occurs.

Fixed revisions of the chips in the sidplay2.ini: some were called R1s when they were R2s, one was called R2 when it was R3.

Galway6581—my hand-fitted guess of Galway chip sound—is removed in favour of the new Trurl_Ext's 6581R2_3384 chip which replicates Galway sound well.

9.5.2008

Fixed more bugs with filter procedure. I was distorting the wrong half-wave of the bandpass waveform. This change helped more songs sound slightly more correct. I still need to validate the phases of the Vhp, Vbp and Vlp to ensure they are right with respect to unfiltered waves.

8.5.2008

Fixed final silly bug with filter procedure. Forgot that Vlp and Vi are at the same level, and I guess I managed to boost the bass levels slightly as a consequence.

Updated sidplay2.ini too with the newest measurements from my C64. Added new R4 chip.

7.5.2008

I redid the patch, adding some missing Makefile.in files and like into the sidplay2 part. It might be compilable now, if you are lucky. (I hate automake.)

6.5.2008

In accordance with the New World Order, I added support for Pulseaudio. I also decided to release the code as V10. The world has suffered V9 for long enough. While V10 is not perfect, it is pretty nice.

5.5.2008

Wow, a long pause. I redid the distortion algorithm again, deciding that the other half-wave of the waveform must not distort at all (at least compared to the other halfwave). The results seem very encouraging, so I published the updated patch immediately. (I really started to hate the "dry" distortion sound that the previous patch was making.)

Oh yeah. I also broke type 3 designer, I'm going to redo the type 3 curve a bit in order to get the "kinked" behaviour better especially at the now-problematic 512, 1536 and 768, 1280 pairs.

28.3.2008

I'll likely try to boost SID emulation for V10 by writing simulation of quadratic mosfet equation in place of the current relatively ad-hoc exponential approximation. I've mostly tweaked params, but I can't seem to find one setting that would replicate every song acceptably.

24.3.2008

Started V10. I tried adding ring modulation support for the ST combined waveform, but it looks like 8580 doesn't do it, and 6581 has some kind of odd bug with this sound, which seems to somehow clamp the oscillator to only repeat a small subset of the full wave. (It only makes a quiet sound.) So I'm going to take it off soon. I need to do something else for 6581.

22.3.2008

I made a small parameter tweak into the .ini file. The purpose was to change the distortion sound a bit for extreme distortion, which had become too noisy in my opinion. Rather than producing the distorted and filtered sound, it mostly scratched. This was noticeable in Miami Vice and Gloria. It's still not perfect, but it's better.

20.3.2008

Another update to V9. I renamed the old "Headroom" variable, and introduced another. The new tunables are DistortionLowpass and DistortionHighpass, and the DistortionRate was kept.

Rate controls how strongly the distortion signal drives the distortion. It's a divider, so smaller value has larger effect.

The lowpass approximates the system capacitance and resists the distortion effect. I believe this value to be around 1000, as I've always found that to sound best.

The highpass reduces distortion for bass sounds, and the tunable currently required for unknown reasons. The smaller the value, the less effect there is per rate, so rate value generally needs to be decreased too.

I noticed some songs have difficulties when switching, so I added the distortion state variables to the filter reset routine. Known bug: it seems sidplay2 doesn't detect 8580 songs, but plays them with 6581 emu. I may fix this for V9 yet.

18.3.2008

I optimized e^x as 2^x, and approximated it with floating point value constructor. The approximation is the laziest ever done, perhaps, but is always accurate within 3 %. This dropped CPU usage by about 8 % on my system.

I added highpass filtering for the source term. I got no theoretical justification for this, apart that with this it works great. I updated V8 again. Sorry. I know this is supposed to be stable... All the changes are in the filter.h.

17.3.2008

Experimenting with tweaking op-amp limits again. Hard Track indicates that the SID has some kind of nonlinearities involved internally that bias the shift effect upwards at least for the kind of signal it is using. I can replicate it, but the cost is destroying the bass performance. The current values are hopefully some kind of compromise.

V9 is such a large improvement over V8 that I have decided to mark it stable. I believe, however, that V10 shall be the Real Thing. In the meantime, happy listening!

16.3.2008

New distortion algorithm. The code is very, very, very slow, but realtime at least on AMD64 X2 4200+. I merged Type 3 straight into ReSID, which in theory allows a new level of accuracy with the distortion. Not all songs work with it yet, and it's basically untuned. However, at least Spaceman Salutes Commodore and Snake Disco seem to have improved markedly, and Nata's filter_a SID also plays correctly.

15.3.2008

I got MMCReplay last Thursday and set up C64 for sampling songs today. I removed all but the the R4AR samplings. I made the old catweasel samplings unavailable.

Because the R4AR chip sounds very different in a real C64 comapred to the Catweasel Mk4, I refitted a new curve for the R4AR, which seems to reproduce many songs quite authentically.

The V8 has incomplete denormal number avoidance code, I think. Sometimes this shows up as sidplay2 CPU usage doubling or even tripling.

14.3.2008

Started V9. Experimenting with highpassing the distoriton term. Initial results seem quite promising. I ripped my computer box open and discovered that the chip on the Catweasel is not R4 but R4AR. Oh the embarrassment. I changed everything R4-related on the page to say R4AR.

13.3.2008

I decided to release V8 as bugfix to V7. The settings are bit on the conservative side in an effort to avoid problems that otherwise crop up. I have especially delicate problem with lead sounds that tend to distort in a very annoying fashion. Earlier I was able to fix this in V6 by flipping the bandpass output, and that was one of its highlights, but unfortunately that road turned out to be a dead-end. This code should still be better than V6 or V7, but it is not the rework of the distortion code I hoped it to be.

11.3.2008

I have updated the filter calculation code to floating point. It seems that for very low FC values, integers are unable to provide the needed resolution and the output becomes heavily quantized to something like 5-6 bit audio. It might be possible to salvage integer arithmetics somehow, but at the moment I'll just take the easy way out and just use floats where needed.

6.3.2008

Started V8 experimental as a minor parameter tweak to V7.

2.3.2008

Busy weekend. I made some nonfunctional changes to V7, exposting the filter at sid.cc to callers through get_filter() method, and fixed a few types.

I also fixed the stereo sid playback code and implemented mono/stereo mix modes that I removed previously. This way the output config does not have to match the SID being played. I killed the left-channel-only and right-channel-only modes, though. I changed force-dualsid mode to place the second SID throughout the address range, apart from the first SID address. This should make every song work...

28.2.2008

Started V7 based on circuit image provided by Lord Nightmare. The system is now nearly perfect. The long road seems almost over, and the finishing line looms ahead.

V7 is large enough improvement that I have decided to mark it as stable after only about day and half of work on it. Sorry for all who just managed to ingrate V6. I've unfortunately redefined the Type 3 filters in light of the new information I have acquired, so that work goes a bit anew. (This is another reason: I wanted to update to V7 before Type 3 gets too established.) Otherwise the changes should be limited to couple of filtering tweaks and small cleanup in libsidplay.

26.2.2008

Released V6. After several hours of listening, I believe it's better than what was before.

New very experimental distortion function that is basically an offsetted parabola. The asymmetry allows the distortion to respond to varying signal levels in somewhat convincing way. Some songs such as Elite reach the other edge, and respond favourably to being distorted there. This doesn't make sense electrically speaking, though. ☹ I think the other edge of the parabola may be further away than given by the current parameters. The only problem is, there shouldn't be an edge of parabola there. Well, whatever.

I'm also experimenting with lowpass filtering the whole distortion term, which may or may not be correct. At this stage, it's still hard to say.

I also added some emulation fixes: sample hold on waveform 0 (this should also do the analog fade thing in a few seconds, but isn't emulated), noise waveform lockup on combined waveforms, and noise register analog fade simulation.

I also spent quite a while trying to get Snake Disco and Spaceman Salutes Commodore to work, but they stubbornly stay out of reach. Bummer.

24.2.2008

Fixed 8580 bandpass phase in V6. I removed the opmin/opmax terms as they seem not required in practice. I also migrated all the work into SVN to ease development and clean the patch up.

23.2.2008

New experimental V6.

I dropped libsidplay's bad resampling code. It now uses ReSID's perfect resampling. I had to trash the beautiful parameters used by ReSID in order to bring the FIR filter CPU consumption down a bit. (Playback was hovering around 80 %,which is completely unacceptable.) The people who designed the filtering did not realize that quantizing the FIR coefficients to 15 bits will set a relatively high noise-floor. (I estimate it to be around -70 dB.) I also decided you probably can't hear much around 18 kHz and above, so I brought filter length down further to save some CPU. I see 40 % as a typical value for AMD64 X2 4600+ after tweaking the filter shorter. What wouldn't I give up for great SID playback...

Happily, I discovered that libsidplay configured two emulated chips always even if the song would output using only one. I instantly disabled the mute chip completely by replacing it with a "nullsid". This halves the CPU usage again, bringing it down to about 20 %.

21.2.2008

Small correctness tweak into distortion term. I correct for the 1023-to-1024 CF kink in my derivate approximation.

20.2.2008

Obsoleted V4, released V5

Injected CF derivate into the distortion term, thus making it behave more like real chips do. Merged the separate distortion terms into one, based on Vhp + Vbp (modelled as inverted). V5 reduces the quality of 8580 emulation slightly by reversing the phase of the bandpass output. This is probably not an audible effect, but it will probably be corrected later on.

19.2.2008

Made another bugfix: for some reason ReSID code adjusted the external filter frequency based on calculated passband, while in the real C64 the filter is fixed to about 17 kHz. This fix reduces the aliasing noise on sidplay2 output slightly and is therefore welcome improvement. Someone should reimplement libsidplay's mixer.cpp. The code omits all the nice and clean resampling code from ReSID and instead uses a simple pick-nearest sampling without properly lowpass filtering the output first.

I also made a few things consts, relabeled things, and updated the distortion function. It's now a capped parabola which starts from the minimum.

I also changed the equation although this breaks 8580. I believe it is best to split the whole stupid distortion stuff in a separate clock_distorted() function, which is called only when distortion is enabled.

18.2.2008

Experimenting now with reversing the polarity of bandpass output. Initial results look really good: two tunables (threshold and fcfactor) could be completely removed without negatively affecting most songs, and positively affecting Dunn's Elite and Theatre Europe. I also mix Vbp in reverse phase now. Eeben's Spaceman Salutes Commodore has a quite different distorted sound for the 1:20 bit that's hard. This definitely did something!

14.2.2008

Happy Valentine's Day! I have added new type of filter definition that I have chosen to call Type 3 filter (I was tempted to call it Type 6581 filter, though) for V5. I have replaced existing filter CF tables in sidplay2.ini with Type 3 definitions.

I also eliminated some filter clipping by killing bandpass if it distorts too hard. This may affect some songs adversely, but fixed the clicking which is even worse a problem.

Argh. mrsid, the author of the splendid OS X sidplay, discovered a bug with type 3 implementation in sidutils: I forgot to include that kinkiness parameter into the final calculation in the end.

13.2.2008

Added bandpass saturation term and tweaked R4 to be a bit brighter overall.

12.2.2008

I have successfully replicated all the CF-to-freq curves of the chips for which I know they have been mapped, using a trivial python program. The simulation is accomplished with 4 tunables, and the match is within 1/100 or so, DAC kinks and all. Of those 4 tunables, 3 becomes fixed when chip type (R3 or R4) is chosen, and only one is variable. Stay tuned for synthetic Galway R3, which should sound just like the real thing!

Also, mrsid (the author of OS X sidplay) discovered a null pointer dereference bug. Thanks!

11.2.2008

Modified the FC calculation. Now it modifieds both the threshold and rate through scaling the source signal. This reproduces even more true SID features.

8.2.2008

I decided to add the FC register value to the threshold to attempt to better simulate the varying sensitivity of the distortion. Some songs are better for it, and some others worse off. Ah well. I'll release this version as V4 because I'm out of ideas to try for now.

6.2.2008

Grue gave me a lot of sample songs to compare V3 to. A lot of them sound discouragingly bad with my patch V3.

I got interested to prepare V4 after chatting with Kevtris. He is aiming to replicate the filter using analog hardware. He has the EE experience that I have sorely lacked and made an infinitely valuable contribution by telling me exactly how the distortion works. V4 is the embodiment of these ideas, and it is also the first real step forwards for months. So far, it's still relatively untuned.

Initial versions in fact crashed during some songs, such as Miami Vice. This is now avoided by clamping the filter outputs, which corresponds roughly to op-amps bottoming out. This does not happen in normal circumstances, merely at extreme distortion that would otherwise send the filter state into la-la land.

8.12.2007

Separate parameters for band-pass (high) and low-pass (low) distortion. This doubles the number of tunables, but seems like a necessary step to open experimentation on this front.

6.12.2007

New distortion algorithm. After I scratched the whole model of substracting filter states from each other. The new model simply calculates distortion by mixing in the filter state terms and input in linear combination. The following rules seem to apply:

1. Vlp term must be about 0.9. More than that, and Jeff's bass sounds lose their edge. Too little, and pretty much everything distorts wrong.
2. Vbp term be something negative like -0.5 or -1 or Vendetta and Meanwhile The Planet doesn't work.
3. Vhp must be small or the distortion loses its "edge".

The new model has one important property above the previous one: it only distorts the other half-wave, the same which can be seen distorted in SID samplings, too. Therefore it makes for more complicated distorted sounds, and better replicates the effects in f.e. Mechanicus.

5.12.2007

Nearly doubled the impact of the bp-hp term. This improves Elite and Shades quite a bit, without seeming to negatively impact other songs.

4.12.2007

Made tunables out of the rest of the distortion parameters, save from one that I just added: low-pass filtering for my crude w0_eff term. This seems to reduce the artifacts on Robocop 3, and generally doesn't seem to harm the songs.

3.12.2007

A new distortion algorithm. A lot of time has passed. I made some improvements that force more distortion out of the model. I feel that this reinstates some of my LEADDIST effects and should be closer to correct model. I'm reasonably happy about how it turned out. This is definitely the best all-around emulation to date.

I also made the distortion tunable via config file, so the thing is sane and doesn't trash 8580 any more.

18.4.2007

Oops, a few months passed as I was busy at work.

I've got rid of the LEADDIST effect, simply accepted that things will not work with it. I hope to be able to replace it with another trick up my sleeve, but so far the new songs and patch probably constitute a step backwards. Maybe.

13.2.2007

Defeat with the masterful TBB songs such as 0_N_0 and Meanwhile The Planet. The 0_N_0 makes a lot of snapping because in a real chip the distortion terms are "filtered" by the other filter components, while mine are filtered by a static ad-hoc estimate. So, the snapping here comes from the LEADDIST term and it doesn't make the right kind of effect.

Meanwhile The Planet makes masterfully precise use of the filter, and there isn't enough distortion for ReSID rendition to be palatable. Thanks to Grue for digging these songs up. Also fixed invalid sidplay2.ini configuration file (was not using R4 curve). This, too, was spotted by Grue.

I suspect Snake Disco might be fixed by accounting for the loss of intensity of sound with the Vbp op-amp distortion. I made some measurements last night and discovered that level of lp and hp outputs dropped dramatically when bp was driven to distortion. I observed loss of up to 20 dB. I probably have to make many more measurements to allow mapping an estimate for this behaviour.

12.2.2007

I added a minor tweak, adding an estimate of the resonance term in the op-amp loading measurement. I also made the bass distortion slightly more prominent, although it still isn't so harsh as the real chip produces.

11.2.2007

I decided the price for fixing RoboCop 3 on 10.2. changes was too high to pay. It put a few dB of distortion back and migrated the distortion back to later time. Other changes in frequency drift implementation however helped a lot even with RoboCop 3, although I do think it still distorts too much. I think overall I'm now better off.

I also moved the distortion term computation into the filter clocking function, so that if output is being requested at some different rate than the filter is clocked, the lowpass filtering would still be computed similarly. This bit me because I had this stupid OptimiseLevel PerformanceHacks accidentally enabled.

10.2.2007

I made the distortion term to start earlier but also more delicate. (Reduced distortion by 6 dB.) This fixed RoboCop_3, SYS4096 and improved locutox's cover of Purple Shades. (Not included on sample set.) I also set bass level to -4.4 dB, because it comes about 1.5 dB louder than treble or bandpass. This is in full contrast to comments in ReSID that claim no level difference exists. Strange, but I can see it plain as day on FFT.

3.2.2007 - 9.2.2007

Creation and tuning the model...

Questions?

Mail to alankila@bel.fi.