RTCM Client

The Micro-Node RTCM and VOTER interfaces are typically used with AllStar in voting/simulcast applications. They MAY be used for ANY repeater interface application, through the chan/voter channel driver.

The VOTER is the original through-hole board designed by Jim Dixon for this application. It is open-source, and the relevant Gerber files and BoM to build it are available.

The Micro-Node Radio Thin Client Module (RTCM) is the commercial version of the VOTER. It uses surface mount parts (SMT), but is functionally equivalent to the original VOTER.

In general, the two terms (RTCM/VOTER) are used interchangeably, as they operate the same, and use the same firmware (mostly, see below).

This page will highlight some of the operational concerns, quirks, bugs, and other items of interest that relate to these interfaces. Much of the information has been gleaned off the AllStar mail list, comes from personal experience, or comes from notes in the firmware source code.

Firmware

The firmware in the RTCM and VOTER is the same, except it is compiled for the specific dsPIC that is installed in each. As such, the firmware is specific to the VOTER or the RTCM, and is NOT interchangeable (it won't boot in the wrong device). Firmware for the RTCM is denoted by "smt" in the filename.

The VOTER uses a dsPIC33FJ128GP802 and the RTCM uses a dsPIC33FJ128GP804.

There are two parts to the firmware, a bootloader, and then the actual firmware file. The bootloader starts when power is applied, and allows you to talk to the dsPIC and load new firmware files over ethernet. If the bootloader is not intercepted by the loading tool, it will continue to boot the current firmware file.

All new boards will need to have the bootloader installed first, followed by a firmware file. You can load a firmware file directly (.hex) in to the dsPIC, but then you will not have any of the bootloader remote loading features.

The current bootloader (.cof file) is available here: https://github.com/AllStarLink/voter/tree/master/voter-bootloader (the -smt file is for the RTCM). It needs to be loaded with a PICKit programmer.

Current firmware (.cry file) is available here: https://github.com/AllStarLink/voter/tree/master/board-firmware . They are loaded with the EBLEX C30 Programmer.

There are multiple "flavors" of firmware available. See below for further explanation of what all the options mean.

The DSPBEW versions have Jim's DSP/BEW feature enabled. Note that due to the size of this feature, the diagnostics menu is NOT available in this version.

The CHUCK versions have Chuck Henderson's RSSI and Squelch modifications enabled. This is probably the version that most user's will want to use in production.

Compiling Environment

If you look in the votersystem.pdf, you will find a procedure to modify and load the bootloader in to the dsPIC of a VOTER board.

Unfortunately, there is an important step missing in that procedure, which is covered below.

In addition, the old links for the MPLAB software are dead, so let's update this info and get you going.

Currently (Feb 2017), you can get the required software from:

http://www.microchip.com/development-tools/downloads-archive

Download MPLAB IDE 32-bit Windows v8.66:

http://ww1.microchip.com/downloads/en/DeviceDoc/MPLAB_IDE_v8_66.zip

http://dvswitch.org/files/AllStarLink/Voter/MPLAB_IDE_v8_66.zip

Download MPLAB C Compiler for PIC24 and dsPIC DSCs v3.31 NOT v.3.25:

http://ww1.microchip.com/downloads/en/DeviceDoc/mplabc30-v3_31-windows-installer.exe

http://dvswitch.org/files/AllStarLink/Voter/mplabc30-v3_31-windows-installer.exe

Optionally, install Windows Virtual PC and XP Mode. This is getting to be pretty old software, so running it under XP Mode may be a good idea, so we can keep it isolated (install it in a virtual machine). It does run fine in Windows 7.

To setup the compile/build environment, follow these steps:

• Run the MPLAB IDE installer. You don't need to install the HI-TECH C Compiler at the end (click no)
• Run the MPLAB C Compiler installer
• Select Legacy Directory Name
• Select Lite Compiler
• Go to: https://github.com/AllStarLink
• Follow the links to: voter --> Clone or Download --> Download Zip. That will get you voter-master.zip which is a download of the whole VOTER tree from GitHub.
• Extract it somewhere (ie. in the XP Mode Virtual PC)
• Launch the MPLAB IDE
• Go to Configure --> Settings --> Projects and de-select one-to-one project mode.

Bootloader

If you need to load the bootloader in to a fresh board, you will need to follow these steps:

• Go to Project --> Open --> voter-bootloader.mcp --> Open (it is in the voter-bootloader folder of the GitHub source)
• Go to File --> Import --> voter-bootloader --> ENC_C30.cof --> Open This step is missing from the original procedure.
• Remove JP7 on the VOTER Board. This is necessary to allow programming by the PICKit2/PICKit3 device.
• Attach the PICKIT2/PICKit3 device to J1 on the VOTER board. Note that Pin 1 is closest to the power supply modules (as indicated on the board).
• If you have not already selected a programming device, go to Programmer --> Select Device and choose PICKit3 (or PICKit2, depending on what you are using).
• Go to Programmer --> Program. This will program the bootloader firmware into the PIC device on the board.

If you want to change the default IP address from 192.168.1.11 in the bootloader:

• Select View --> Program Memory (from the top menu bar)
• Hit Control-F (to "find") and search for the digits "00A8C0".
• These should be found at memory address "03018".

The "A8C0" at 03018 represents the hex digits C0 (192) and A8 (168) which are the first two octets of the IP address. The six digits to enter are 00 then the SECOND octet of the IP address in hex then the FIRST octet of the IP address in hex.

The "0B01" at 0301A represents the hex digits 0B (11) and 01 (1) which are the second two octets of the IP address. The six digits to enter are 00 then the FOURTH octet of the IP address in hex then the THIRD octet of the IP address in hex.

Once you have modified the address to your desired IP, follow the programming procedure (above).

Firmware

To compile the firmware (if you want to make custom changes):

• Go to Project --> Open --> navigate to board-firmware and open the .mcp file for the flavor of firmware you want to compile. They are in the board-firmware folder of the GitHub source.
• NOTE: the .mcp files with the "smt" suffix are for the RTCM (built with SMT parts). The non-smt files are for the ORIGINAL through-hole VOTER boards. The difference is that the VOTER uses a dsPIC33FJ128GP802 and the RTCM uses a dsPIC33FJ128GP804.
• Go to Project --> Build Configuration and select "Release". This may not be necessary (I don't believe that option is used in the firmware), but it removes the compiler option of __DEBUG being passed, so theoretically it would build "normal" firmware.
• Go to Configure --> Select Device and choose the appropriate device for your board from the Device list. If you don't select the right one, the board will not boot. Select dsPIC33FJ128GP802 for the VOTER, and dsPIC33FJ128GP804 for the RTCM.
• Now, if you go to Project --> Build All it should compile everything and show you "Build Succeeded".

A .cry file should be in the board-firmware folder. You can load this with the ENC Loader.

There is also a .hex file in there that you could load with a programmer... but that would wipe the bootloader... so don't do that.

Chuck Squelch

If you want to enable "Chuck Squelch", open the HardwareProfile.h and un-comment #define CHUCK.

You may also want to go down to Line 291 in Voter.c (right click on the window, go to Properties --> "C" File Types --> Line Numbers) and tack on CHUCK after 1.60 (the current version number) so that when you load this firmware, the version will be shown as 1.51CHUCK, and you'll know that Chuck Squelch is compiled in. We should probably make that more automagic in the future...

DSP/BEW

If you want to compile the DSPBEW version, open the DSPBEW project file instead.

DSP/BEW Firmware Version

DSP BEW Firmware is mutually exclusive with the diagnostic menu. There is not enough space for both, if you load the DSP/BEW firmware, you will NOT have a diag menu.

BEW stands for Baseband Examination Window.

Typically, the discriminator of an FM communications receiver produces results containing audio spectrum from the "sub-audible" range (typically < 100 Hz) to well above frequencies able to be produced by modulating audio. These higher frequencies can be utilized to determine signal quality, since they can only contain noise (or no noise, if a sufficiently strong signal is present).

For receivers (such as the Motorola Quantar, etc) that do not provide sufficient spectral content at these "noise" frequencies (for various reasons), The "DSP/BEW (Digital Signal Processor / Baseband Examination Window)" feature of the RTCM firmware may be utilized.

These receivers are perfectly capable of providing valid "noise" signal with no modulation on the input of the receiver, but with strong modulation (high frequency audio and high deviation), it severely interferes with proper analysis of signal strength.

This feature provides a means by which a "Window" of baseband (normal audio range) signal is examined by a DSP and a determination of whether or not sufficient audio is present to cause interference of proper signal strength is made. During the VERY brief periods of time when it is determined that sufficient audio is present to cause interference, the signal strength value is "held" (the last valid value previous to the time of interference) until such time that the interfering audio is no longer present.

The DSP/BEW feature is selectable, and should not be used for a receiver that does not need it.

A note on the Motorola SLR5700 per VE7FET. DSP/BEW definitely makes a difference on what AllMon reports for the signal strength of received signals. It is less reactive when DSP/BEW = 1 than with it set to 0. We'll have to see what the real world trials show. It might be better to leave it off.

Chuck Squelch

"Chuck Squelch" are a couple firmware changes made by Chuck Henderson, WB9UUS.

Pre-compiled firmware versions including this option are available on GitHub. See above on how it is enabled/compiled, if you are rolling your own firmware modifications.

One of the changes fixes an issue with weak signals producing RSSI readings all over the place. It is caused by a 16 bit value that was overflowing (it is the RSSI change in the firmware). It results in rock-solid RSSI values being reported, even on barely or non-readable signals. This change will likely be rolled in to a permanent fix in a future firmware release.

The other firmware change changes how the squelch responds (looks at the noise in the last two audio samples) and makes the "Micor squelch" action work better.

You may also want to consider the following changes in /etc/asterisk/voter.conf:

;Comment out:
;thresholds =

;and set:
linger = 0

Squelch Issues

• If anyone is off frequency a little bit, that will make the voice talk off worse. Double check that the repeater and the users are all on frequency.
• Don't use narrow bandwidth on the repeater receiver.
• Make sure that the discriminator audio is not rolled off even a little bit at the high end. There should not be resistors in series or capacitors to ground between the discriminator chip output pin and the voter board input, for best results.

RTCM/VOTER LED's

RX LED on the RTCM/VOTER will flash (same rate as ACT LED) if you have External CTCSS enabled, and the received signal has the wrong (or no) valid PL.

RTCM/VOTER Bandwidth

Assuming ulaw... The RTCM will require approximately 80kbps of bandwidth for continual usage.

Debug Options

The VOTER/RTCM firmware supports some additional debugging information that can be turned on.

From the source code, the different Debug Options are listed as:

1 - Alt/Main Host change notifications
2 - Ignore HWlock (GGPS only)
4 - GPS/PPS Failure simulation (GGPS only)
8 - POCSAG H/W output disable (GGPS only)
16 - IP TOS Class for Ubiquiti
32 - GPS Debug
64 - Fix GPS 1 second off
128 - Fix GPS 1 month off (WTF,O??)

Not sure what they all do, but that is what they are. Here are the most common ones used:

• "Alt/Main Host change notifications" shows when the connection to the Asterisk server changes state.
• "IP TOS Class for Ubiquity" marks the IP headers from the RTCM/VOTER TO the network with ToS C0/DSCP 48 (UBNT shows this as 802.1p Class 6 (Voice <10mS

latency). Other sources show this as a Network Control TOS.) If you enable this, you also will want to have "utos=y" in your voter.conf to mark the packets from the network TO the RTCM/VOTER. We will probably change this option to mark the packets by default in a future version.

• "GPS Debug" will print NMEA or TSIP debug strings from the connected GPS.

The way this works is you add together the options you want to enable.

Want to enable GPS Debug and IP ToS, set debug to 48.

Just want to turn on GPS Debug, set debug to 32.

Just want ToS turned on, set debug to 16.

Un-documented Menus/Features

• Menu 96 - If firmware compiled with DUMPENCREGS enabled, Menu 96 looks like it dumps the registers from the ENC (Ethernet) chip?

• Menu 111 - show the "hidden" option values (normally they should all be 0).

• Menu 11780 - set the "Elkes" value? No idea what this does.

• Menu 1103 - set the "Glaser" timer value? No idea what this does.

• Menu 1170 - set "Sawyer Mode". No idea what this does, it was a modification Jim made for Tim Sawyer, but Tim doesn't recall what it was for. :)

TX Level Setting

• Send 1kHz@3kHz on-channel in to the RX
• Set the RX up so it reads 3kHz deviation on the RTCM's built in console meter
• Now set the TX level pot to get 3kHz out of the transmitter (No PL)

Now change the modulation from 1kHz tone to 800Hz followed by 1.8kHz and verify that the deviation level doesn't change as the tone frequency changes. Changing levels indicates a pre/de-emphasis issue.

If using PL you have to account for that deviation, unless you filter it out with your IFR (test set).

GPS

The RTCM/VOTER will work with most GPS available. It requires either NMEA or Trimble TSIP binary data. It only receives data from the GPS (GPS TX), it does not send anything to the GPS.

The firmware is specifically written to talk to Trimble Thunderbolt receivers using Trimble's TSIP binary data interface, however, other Trimbles GPS receivers that talk TSIP are generally compatible.

NMEA Sentences

If you are using an NMEA GPS (as opposed to a Trimble using the TSIP binary interface), the RTCM/VOTER is looking for the following NMEA sentences:

$GPGGA
$GPGSV
$GPRMC

GPS Lock

The GPS led will go solid regardless of the connection LED. That has to happen or it won't connect to the Asterisk server.

It is not unusual for it to take up to 20 mins to get a GPS lock LED (ie. using a Trimble Thunderbolt) after any reboot.

GPS De-sense

If you are having odd loss of lock issues, consider you may have interference to your GPS antenna from strong RF nearby. A note from Jesse Lloyd:

I also had crazy problems with poor signal on my GPS when I set it up sitting in a window, and once installed at site I had the GPS antenna maybe 6 ft from the VHF antenna, and after some troubleshooting found it was getting swamped with RF and loosing lock. 

I found the debug setting of 32 useful in the RTCM, you can see a hex output of the GPS status.  

GPS Issues

Trimble Thunderbolt

You may find your Trimble Thunderbolt is showing the incorrect date at the moment. It could be showing the year as 1997. This is due to the date in the Thunderbolt being reported incorrectly.

This can cause some of your voting receivers to not connect, if they are used with other GPS in your system. If you have ALL Thunderbolts, or NO Thunderbolts, you are probably fine. If you have ALL Thunderbolts, the date/time is probably wrong, but they will ALL be wrong, so they will connect.

GPS Time is a continuous counting time scale beginning at the January 5, 1980 to January 6, 1980 midnight. It is split into two parts: a time of week measured in seconds from midnight Sat/Sun and a week number. The time of week is transmitted in an unambiguous manner by the satellites, but only the bottom 10 bits of the week number are transmitted. This means that a receiver will see a week number count that goes up steadily until it reaches 1023 after which it will “roll over” back to zero, before steadily going up again. Such a week rollover will occur approx. every 20 years. The last week rollover occurred in 1999 and the next one will be in 2019.

The Thunderbolt manual states:

The ThunderBolt adjusts for this week rollover by adding 1024 to any week number reported by GPS which is less than week number 936 which began on December 14, 1997. With this technique, the ThunderBolt will provide an accurate translation of GPS week number and TOW to time and date until July 30, 2017.

As such, the Trimble Thunderbolt has a firmware issue with the GPS Week rollover that manifested itself on July 30, 2017, causing the date to become incorrect.

We have added a brute-force fix starting in RTCM/VOTER firmware >=1.51. It fixes the Thunderbolts, we have not done extensive testing to see how it affects other TSIP receivers.

Garmin

Garmin 18x LVC Wiring Issues

If you have issues with your GPS 18x LVC not talking to the RTCM/VOTER, it may not be hooked up to the RTCM correctly.

The Garmin pin labeling is backwards to what you may think. See below. You probably need to swap pins 6 and 14.

RTCM   	                 GPS 18x LVC      
6 GRX 	<-- Rx Data 	 6 Green  
7 GPPS 	<-- Pulse Output 1 Yellow  
8 GND   Ground 		 3 Black  
8 GND   Ground 		 5 Black  
13 +5V 	-->Vin 		 2 Red  
14 GTX 	--> TX Data 	 4 White

Log into the RTCM and do 98 and you should see something like this:

Current Time: Sun  Apr 20, 2014  04:37:02.820
Last Rx Pkt System time: 04/20/2014 03:55:35.580, diff: 2487260 msec
Last Rx Pkt Timestamp time: 04/20/2014 03:55:32.064, diff: 3515 msec
Last Rx Pkt index: 160, inbounds: 1

Garmin and the RTCM

Beware when buying newer Garmin GPS's to use with the Micro-Node RTCM.

The RTCM expects a 5V PPS signal.

Newer Garmin's (GPS 18X, 18X LVC, etc.) MAY NOT output 5V, and can cause issues. Check the Garmin datasheet.

The VOTER is designed to accept both 3.3V or 5V signals, and should work fine.

No GPS/Mixed Mode

You do not need a timing source to use an RTCM if you don’t want to vote. That’s called Mix Mode.

Having the PPS Polarity set to 2 tells the RTCM you do not have a GPS. That forces the RTCM to become a mix (non-voting) client. Turn it on, save it, and restart. A simple voter.conf would look like this:

[general]
port = 667
password = BLAH

[1000]
Site1 = pswrd1,master,transmit
Site2 = pswrd2,transmit

Mixed Client Error

"I am getting this error in Asterisk":

WARNING[2368]: chan_voter.c:4511 voter_reader: Voter client master timing source mobile1 attempting to authenticate as mix client!!

A mixed client error means the voter.conf file is expecting an RTCM to try and connect with a GPS IP packet (ie. has master in it in voter.conf), but the RTCM isn't sending a GPS IP packet. So its a mismatch between voter.conf and Option 10 in the RTCM.

If you want to use a mix client (non-voted), make sure that receiver's configuration line in voter.conf does not have the master option set.

GPS Debug

To turn on GPS Debugging, set the Debug Option Level in the RTCM/VOTER to 32.

See #Debug Options levels for more information on how this works.

Trimble Debug Status Decoding

The VOTER/RTCM when in TSIP mode (Trimble), assumes it is a Trimble Thunderbolt and is looking for two packets:

• 0x8F-AB - Primary Timing Packet
• 0x8F-AC - Supplemental Timing Packet

Grab a copy of the Thunderbolt User Guide: http://leapsecond.com/pages/tbolt/Thunderbolt-2012-02.pdf

Pages 78-83 are the important ones.

Packet 0x8F-AB is what grabs the timing information, and packet 0x8F-AC is what it looks at for everything else (including debug).

The debug string that the VOTER/RTCM reports:

	printf("GPS-DEBUG: TSIP: ok %d, 9 - 14: %02x %02x %02x %02x %02x %02x\n",
					happy,gps_buf[9],gps_buf[10],gps_buf[11],gps_buf[12],gps_buf[13],gps_buf[14]);

So, the 1 after ok is the "happy gps" flag. The other 5 bytes are supposed to be bytes 9-14 from the packet... sort of. They are bytes 9-14 of the buffer, but they are actually bytes 8-13 of the binary message.

Looking at the message structure, bytes 8-13 are:

• 8-9 Critical Alarms
• 10-11 Minor Alarms
• 12 GPS Decoding Status
• 13 Disciplining Activity

Therefore, the messages you are seeing break down as follows:

GPS-DEBUG: TSIP: ok 1, 9 - 14: 00 00 00 00 00 00 - everything is good in the 'hood, Doing Fixes, Phase Locking

GPS-DEBUG: TSIP: ok 0, 9 - 14: 00 00 00 18 08 06 - not happy, Not Tracking Satellites, Not Disciplining Oscillator (0x18 --> 0b000000011000), No Usable Sats, Inactive

GPS-DEBUG: TSIP: ok 0, 9 - 14: 00 00 00 08 08 05 - not happy, Not Tracking Satellites, No Useable Sats, Compensating OXCO (holdover)

GPS-DEBUG: TSIP: ok 0, 9 - 14: 00 00 00 00 00 05 - not happy, Compensating OXCO (holdover)

GPS-DEBUG: TSIP: ok 1, 9 - 14: 00 00 00 00 00 08 - happy, Recovery Mode

GPS-DEBUG: TSIP: ok 0, 9 - 14: 00 00 00 00 00 04 - not happy, Initializing Loop Filter

The GPS is flagged as NOT HAPPY in TSIP mode if ANY of the following are TRUE:

• If GPS Decoding Status is anything other than "Doing Fixes".
• If Disciplining Activity is not Phase Locking or Recovery Mode.
• Any Critical Alarms.
• Any Minor Alarms.

Ubiquity ToS

See https://help.ubnt.com/hc/en-us/articles/205231750-airMAX-How-is-QoS-and-prioritization-handled-by-airMAX-

So, if we set ToS/DSCP in the header to C0, then Ubiquity (and other gear watching ToS) should prioritize the packets. This sets the DSCP to 110 000 aka 48. UBNT shows this as 802.1p Class 6 (Voice <10mS latency). Other sources show this as a Network Control TOS.

There is an option in voter.conf to turn this on (utos=y). However, this only controls packets being sent from Asterisk TO the RTCM.

If you want to tag packets from the RTCM TO Asterisk, you need to set the RTCM debug option level to 16 (see #Debug Options for how this works).

At some point we should probably change the default behaviour of the RTCM firmware to mark the packets and use the debug setting to disable ToS. That change (if you wanted to compile your own firmware) would be:

Change line 90 in IP.c

From:
#define IP_SERVICE         ((AppConfig.DebugLevel & 16) ? 0xc0 : (IP_SERVICE_ROUTINE | IP_SERVICE_N_DELAY))

To:
#define IP_SERVICE         ((AppConfig.DebugLevel & 16) ? (IP_SERVICE_ROUTINE | IP_SERVICE_N_DELAY) : 0xc0)

This changes the conditional expression from if the debug level is 16 to mark the packets with DSCP 48 to if the debug level is 16, mark the packets routine (DSCP 0).

RTCM Simulcasting

The RTCM/VOTER boards do support simulcasting, however, there are a bunch of quirks that one needs to be aware of.

Radio Hardware

For best results, you should use all identical RF equipment between your voting RX sites and simulcast TX sites. If you don't, you can end up with strange audio artifacts when different receivers are used, and other strange audio issues when different transmitters are used.

9.6MHz Oscillator

If you want reliable simulcast with an acceptable level of overlap warble, you MUST inject a GPS-disciplined or OXCO 9.6MHz signal to the clock of the RTCM's, in lieu of their stock internal crystal.

Otherwise, the inherent internal clock jitter of the RTCM's will cause CTCSS warble, etc. in areas where your transmitters overlap. The 1 PPS feed that you normally feed the RTCM's is acceptable for voting, but naturally, a typical crystal-driven clock will slightly drift in between the pulses every second due to inherent jitter, etc., and that ain't to good for situations like simulcast which demand sample-accurate performance.

Here are some comments from Joe, KC2IRV, on the subject. He has also created his own wiki, specific to simulcast issues, you can find it at: http://rtcmsimulcast.wikifoundry.com/

I felt I needed to let those on this mailing list know my findings with simulcasting with the RTCM units since I have received a great deal of help and information from people on this list.

For the past week I have had a two (2) site simulcast system up and running on UHF using the RTCM units. I have done a great deal of testing and proving on the bench and it is so far working beautifully.

The one issue that was made apparent to me early on was that in order for the RTCM's to be suitable for simulcast, the processor inside it needed to be clocked to a more accurate source. This source is supposed to be the Programmable Clock Generator Module in order to produce a GPS locked 9.6 MHz reference for the processor. Unfortunately this unit is not currently available.

Knowing this, I started to go down the road of possibly designing my own, but stumbled upon a cheap, plentiful supply of Symmetricom 9.6 MHz sinewave OCXO's. I took a chance and ordered a few to experiment with to see if the RTCM would be able to use this in place of the supplied crystal for a clock. I found they did, with that I decided to modify the RTCM with an SMA connector just above the audio adjustment pots and use this OCXO. I trimmed the frequency on the units to match each other and used them to produce the processor clock.

After a week of testing, I have found the results in the overlap areas to be exactly as I would expect them on a public safety/commercial simulcast system. While in an overlap region where both transmitters are within 3dB of one another, to the ear you hear no audio phase wandering. Of course , this has only been a week of testing so I wouldn't call it case closed yet.

I will say that if this system remains this way after testing for 6 months to a year than using a 9.6 MHz OXCO is a viable alternative to achieve the needed precision and accuracy needed for the processor clock to make it suitable for simulcast.

Comments from Jim Dixon on the issue:

You have to use something that will take the precise 10MHz (or whatever it is) from the GPSDO and PLL (phase-locked) convert it to 9.6MHz (square wave at 3.3V for the CPU clock).

We had to use 9.6MHz mainly because of the DAC in the dsPIC instead of 10MHz. The options for various divide/clock radios in the part are RATHER limited. As I recall, there wasn't even a way of getting the necessary 16kb/s sample rate on the ADC from 10MHZ, either.

Micro-Node RTCM Clock Issue

James, KI0KN, had some strange-ness with some of his RTCM's when used for voting. As soon as he changed menu item 10 on the RTCM to either a (1) or a (0) instead of (2), he instantly got this on the RTCM console:

04/05/2016 18:44:13.660  Lost GPS Time synchronization
04/05/2016 18:44:13.660  Host Connection Lost (Pri) (10.16.1.240)

And it sits there forever and never re-establishes connection to the host. The issue was finally traced to a bad batch of 9.6MHz crystals that affected a small run of the RTCM's. The issue was eventually resolved by Micro-Node, but we'll document it here just for record keeping.

James comments to the list:

Well, after many of you offered your time and thoughts on my voter problem, Jim, WB6NIL, graciously donated a couple of hours of his time to remotely help me and he uncovered the problem.

All 5 of my RTCMs were purchased within the last 6 months, and I suspect all 5 have the same problem (will verify that today). The problem is that the microprocessor crystal is running too fast! There is 9.6Mhz crystal that drives the MPU, on the particular unit that Jim helped me diagnose, it's running 2.5 khz too fast.

There is a sanity check in the firmware that makes sure the correct number of samples were taken in the last second (it's supposed to be 8000) and fails if the sample is incorrectly sized (mine is taking 8003 samples). Jim helped adjust the code to be more tolerant of the sampling error and my whole system instantly worked.

I have since tested it with ALL my GPSs (they all work great!). So I now have everything working, including the voting.

I haven't contact Micro-Node about it yet. Jim told me that voting should work fine but this clock error would probably not be acceptable for Simulcast (something we have no plans on doing as of now). Since mine are deployed at mountain top sites, I'll probably pursue a crystal that is running at the correct frequency and use "standard" firmware so that if the decision is ever made to play with simulcast, I won't be right back in this same boat.

Thanks to all that helped out, and a HUGE thanks to Jim for taking the time to troubleshoot this.

BTW- This clock error will NOT effect the RTCM in mix client mode, it only affects it in voting or simulcast mode!

In voter.c, the stock line is:

if ((samplecnt >= 7999) && (samplecnt <= 8001))

it needs to be changed to something like:

if ((samplecnt >= 7999) && (samplecnt <= 8003 ))

That was just enough to make my first RTCM work. I've had a chance now to check a few more of my RTCMs on the bench this morning and the crystal frequencies are kind of all over. The next one I had on the bench was 5.2Khz fast. The above code was still not enough to fix it (as would be expected) so it looks like the quality control on whatever manufacturer that crystal is, isn't very good (or a low tolerance crystal is being used). I am going to pursue crystal replacement to get a highly accurate, stable, on-frequency crystal in there as my first choice. Changing the firmware 5 different times doesn't seem like the right answer.

Jim put that sanity check in there for a reason. Working around it may allow the device to work, but to me, it seems getting the hardware operating the way it was supposed to originally is the better answer. I will follow up with Micro-node and let you all know how it goes.

After following up with Micro-Node:

They were all off, some were off a LOT. I talked to Mark @ Micronode. He told me there were 25 units that made it out the door with a crystal from an unapproved source and that was quite likely the reason for the problem. He gladly sent me 5 new crystals to replace (he offered to replace them himself if I sent the units in, but I am comfortable doing it myself to save the time and postage).

I haven't received them in the mail yet to show that the problem is fixed, but I will post here when that happens. He was great to work with and had no issue getting things set straight. If you are having the same issue and your RTCMs were bought about the same time, I'd suggest you contact him and have him help you resolve the issue!