Parallels Desktop VS VMWare Fusion – deleting multiple snapshots

I have used both Parallels Desktop and VMWare Fusion to run Windows, largely for Windows software development, on a Mac, over the past several years.  There have been a number of reviews comparing them, which I won’t repeat.  Parallels Desktop allowed much better hardware configuration and much better keyboard mapping a couple of years ago, which is why I have been using it over the past couple of years (newer versions of VMWare Fusion may have improved that — I don’t know).

One of the biggest advantage of using Virtual Machines for software development (with either Parallels Desktop or VMWare Fusion) is the ability to easily create snapshots.  When you are doing development, you often need to install or update software components, and never know when you may destabilize your system.  The ability to easily create system backups is similar to commits in source code control, in that you can easily save multiple states that you can go back to if you need to.  As a result, I tend to create a LOT of snapshots.

My biggest complaint about Parallels Desktop in comparison to VMWare Fusion is that it does not allow you to delete multiple snapshots at once.  Deletion of one snapshot (with either Parallels or VMWare) can easily take five to ten minutes or longer.  If you have to delete thirty or forty snapshots, clicking on one, waiting ten minutes, clicking on the next, waiting ten minutes, it can get very time consuming.

VMWare Fusion has had a much better solution for this problem, and is much better than Parallels Desktop in this regard.  In Fusion, you merely select all of the snapshots you want to delete, and then select “Delete”.  It may take a while, but you don’t have to manually select each snapshot, one at a time, delete it, and wait.  It can run overnight, and when you come back, the work is all done!  Alas, Parallels Desktop does not allow you to select more than one snapshot to delete at a time.

Recently, I submitted a request to Parallels about my problem, and to my delight (and surprise), their technician called me on the phone the next day to help me.  Ultimately, they led me to what seems to be a solution, but not without a couple of mis-steps, so I wanted to document my experience for anyone else having this problem.

One thing about snapshots: they are NOT backups.  The virtual disks with either VMWare or Parallels are MUCH more fragile than a regular hard drive controlled with a journaled format by a modern operating system (Windows, Linux, or OS X).  I have had both VMWare and Parallels disk images become corrupted fairly often, which results in ALL of your snapshots (and everything else) being lost.  If you use VM’s, you MUST backup the ENTIRE VM FILE, just as with source code control.  That also has the advantage of backing up all of your snapshot history.  Before doing anything I describe here, be sure you have at least one backup (and preferably more, on different disks) of your VM.  As I mention, my VM DID become corrupted, and I had to use a backup.  You have been warned.

I had backed up my entire VM with all of my old snapshots, and I just wanted to remove all of the old snapshots from my current, working copy.  I had a virtual hard disk with about 100 GB of files, but the VM had swollen to about 450 GB with all of the snapshots.  Parallels has a “Delete snapshots with children” option, which I had used to delete any “branches” I had (usually when I had to go back to a working configuration).  Thus, I just had one long line of snapshots in my Parallels VM.

The technician directed me to use  the prl_disk_tool merge option.  He directed me to the http://kb.parallels.com/9165 page.  He had me open a terminal session, and copy

prl_disk_tool merge --hdd

onto the command line.  Then, he had me open Parallels, and in the Virtual Machines List, right click on the offending VM and select “Show in Finder”.  Then, right click on the VM file in Finder and select “Show Package Contents”.  Then (and I didn’t know you could do this), click on the large .hdd file and drag it into the Terminal window — that copied the full path of the .hdd file, properly escaped, onto the end of the command line I was building, which left me with a command line like

prl_disk_tool merge --hdd /Users/nachbar/Documents/Parallels/Win 7.pvm/Win 7-0.hdd

Then, press Enter.  I was immediately presented with an operation progress starting at 1%, and slowly increasing to 100% over about 90 minutes.  At the same time, the size of my VM dropped, and the free space on my physical hard drive increased, by about 300 GB.  But that only partially fixed the problem.

I was then able to run my smaller VM with the merged snapshots, but Snapshot Manager showed all of the snapshots still there.  I emailed them back, and they directed me to the page about snapshot manager and deleting snapshots there.  I was able to delete them one at a time, and it was a lot faster (five to ten seconds each).  That page also described “Delete with children”, and so I thought I would use that, figuring that I would be warned if it was going to delete my current VM state.  Big mistake.  I was almost immediately met with the “Unable to Connect to Parallels Service” error described at http://kb.parallels.com/8089 .  The “Deleting Snapshot” animation continued, however, and I allowed it to run, figuring that if I killed it I would certainly corrupt my VM.  However, after 14 hours, I really had no choice but to kill the Parallels application.  I rebooted, and to my surprise, the VM booted (with a message from Parallels that it had recovered from a serious error).  However, all of the snapshots were still there, and now I could not delete any of them, being met with the “The configuration file you specified is invalid” error.  Thus, still a kind of VM file corruption.

So, I went back to my VM file backup, and started over.  I again ran prl_disk_tool merge, and this time I just deleted the snapshot zombies one at a time, each taking only about 5 seconds (since the underlying snapshots had already been merged).  About half way through, I still had some trouble — I got the “Unable to Connect to Parallels Service” message, and Parallels hung and had to be Force Quit.  Restarting Parallels produced the “Unable to Connect to Parallels Service” error, so I followed the instructions in http://kb.parallels.com/8089 to restart the Parallels Service.  I also got the “Parallels has recovered from a serious error” message.  I was then able to proceed with deleting more snapshot zombies until I had removed all of them.  Now, my VM is down from 450 GB to about 117 GB, and the VM seems faster (at least on startup, which used to be quite slow).

So, clearly a kludge, and still involving a Force Quit and Serious Error recovery, but ultimately I got to a much smaller VM.  I would note that I usually shut down my VM before doing these procedures.  I’m not sure if it is needed, but it seemed reasonable.

Hopefully, Parallels will catch up to where VMWare has been for many years and allow us to select multiple snapshots to delete at one time.  In the meantime, you may want to try this ON A BACKUP OF YOUR VM if you have a lot of snapshots and a bloated VM.

Configuring AudioCodes MP-112 VoIP Gateway for Fax with Asterisk

I am setting up an Asterisk/Elastix system to work with a Cox PRI circuit, and I needed a gateway for managing faxes.  I had previously used the AudioCodes MP-202, which was fairly easy to set up, but that one is no longer available.  Its replacement appears to be the MP-112.  However, the setup is far more complicated, largely because the MP-112 has a lot more capability, but also because the defaults for the MP-112 were not helpful for my application.

This is a very flexible computer and router, capable of mastering DHCP, acting as a firewall, etc.  However, it is set by default to intercept any faxes and route them via T.38.  T.38 only works if the receiving system is expecting T.38 (Asterisk is not capable of taking T.38 and turning it into a regular fax).  If you don’t disable the T.38, simple faxes might squeak through, but even full page faxes will get intercepted.  The symptom will be that only part of the fax page goes through.

Configuring the MP-112 requires first configuring it to register as a SIP extension, and then configuring it to send calls to the Asterisk server (as the “proxy”), and then disabling fax detection so that fax calls go through as regular voice calls.

You connect to the MP-112 via the web interface.  The default IP is 10.1.10.10 .  The default username is Admin, and the default password is Admin .   Note that I (and others) have had trouble with Safari caching a cookie or something and having trouble authenticating.  I had better luck with Firefox or Chrome.  Note that, with Chrome, when I pressed the ‘t’ key, the whole page cleared!  If I needed a ‘t’, I was able to get it by pasting the text in.

For troubleshooting, you can view the MP-112’s log from the Status and Diagnostics tab, under the Status and Diagnostics folder within that tab, as “Message Log”.  Note that, as long as you are on that “Message Log” tag, the log is running.  I was able to select all and then copy the log to TextMate to review it.

Also under Status and Diagnostics/Gateway Statistics is “Registration Status”, which tell you whether your SIP “phones” have registered properly, as well as lots of other diagnostics.

Whenever you want to save the changes you have made on a page, click “Submit”.  You should do that for every page before leaving the page.  To save the changes permanently into flash (so they are not lost when the device is unplugged), press the “Burn” button.

There is lots of documentation on these very complex devices on the AudioCodes website.  This is just a “quick-start” for the common use case (for me) of using the AudioCodes to connect a fax machine or analog phone as a SIP extension on an Asterisk system.

Before you start, go to the Management tab, Software Update/Configuration File — from here you can download the configuration file from the MP-112 that describes the factory default.  If you ever want to go back, you can also upload a configuration file.  Once you get it working, after burning your configuration into the flash rom, save a configuration file so you can always go back.  With the hundreds of configuration settings for this device, the configuration file also gives you an easily-visualized view of the changes you have made.

AudioCodes nicely documents the configuration file format.  In the configuration file, there are sections indicated by [section name].  Those sections are only for human consumption, and do NOT have any effect on the function of the configuration file.  Thus, you can change the configuration file, and don’t have to worry about the sections.  The exception is the table configuration.  Tables are bounded by [ TABLENAME ] … [ TABLENAME ].  The tables have a very specific format, and for tables, the “section” names DO matter.  Also, if a configuration setting is not mentioned in the config file, generally a documented default is used.

When making changes to a page, be certain to hit the “submit” button on EACH page.  Don’t forget to hit “burn” when you are done.

First, you may want to go to the Configuration/Network Settings/IP Settings, and set the IP address,netmask, and default gateway.  Alternatively, the MP-112 will read its information from a DHCP server, if one is available, if DHCP is enabled on the device.  If you use DHCP, you will need to figure out the IP address assigned, so you can access the web interface.  If DHCP is available and enabled, it seems to override the setting under IP address.

The DHCP option is under Configuration/Network Settings/Application Settings/DHCP Settings/Enable DHCP, but it is hard to make it “stick” — Section 10.1.8 of the User Manual indicates that you have to do a reset with the reset button — however, that did not work for me.  I had to click “Submit”, and then “Burn”.  Then, unplugging and replugging the power cord worked to cause DHCP to be used.

Once you have IP configured, you have to get both ports to register with the SIP server (Asterisk) and route calls.

Regarding the call routing, the MP-112 can use a SIP proxy, a routing table, or both.  Unfortunately, the defaults do not favor either option.

Under Configuration/Protocol Configuration/Proxies-IPGroups-Registration/Proxy & Registration/Use Default Proxy, set this to Yes.  This is “IsProxyUsed” in the config file.  In the Proxy Sets Table (the button is on the same page), put the IP address of your Asterisk server under Proxy Address, and UDP under transport type.  You only need the one proxy, so just fill in the first line.

You need to set the authentication to Per Endpoint, so that each port can register. Set Configuration/Protocol Configuration/Proxies-IPGroups-Registration/Proxy & Registration/Authentication Mode to “Per Endpoint” Then, you need to set Configuration/Protocol Configuration/Proxies-IpGroups-Registration/Proxy & Registration/Enable Registration to Enable.

Open the Authentication page:Configuration/Protocol Configuration/Endpoint Settings/Authentication and enter the SIP username and password for each endpoint.  Each endpoint is configured as a separate extension.  In Elastix/FreePBX, the username is typically the extension number, although that’s not the most secure way to configure your PBX.

On Configuration/Protocol Configuration/Endpoint Number/EndPoint Phone Number, you can set the extension number for each of your ports.  If you put 1-2 under “Channels”, the “Phone Number” is the phone number for the first FXS port, and the phone number for the second is the phone number for the first, plus one.  If you put “1” under channel in the first line, and “2” under channel in the second line, you can assign independent extension numbers for each port.  Press “Register” to register with the Asterisk server.  You can check whether registration worked under Status & Diagnostics/Gateway Statistics/Registration Status, and/or from the CLI interface on Asterisk using SIP SHOW PEERS.

Make sure that the MP-112 shows “Registered” for your ports before you move on.

Next, you need to set Max Digits in Phone Num (Configuration/Protocol Configuration/Protocol Definition/DTMF and Dialing/Max Digits in Phone Num) to something like 30 (or at least a number no less than the greatest number of digits you want to be able to dial).  If you don’t, the default is 3, and every time you enter three digits, the MP-112 will try to connect the call using only those three digits.

Now, if SIP registration is working, you should be able to make phone calls into and out of your MP-112.  Test that before working on the fax setup.  You can use the Status & Diagnostics/Status & Diagnostics/Message Log to see the SIP messages and additional debugging info, but be certain NOT to leave that as the active tab in your browser.

Now, for the fax handling:

On the configuration/Protocol Configuration/Coders and Profile Definitions/Coders page, I left only G.711A-law and G.711U-law as the codecs being used.  If you want to use different codecs, enter them here, but some compression schemes may not work well transporting faxes.

To avoid the T.38 fax interception described above, you will need to use “Fax / Modem Transparent Mode”, as described in section 8.2.6.2.7 (page 252) of the MP-112 users manual.  This page shows the parameters you want:

First, set IsFaxUsed = 0 ; This is at Configuration/Protocol Configuration/Protocol Definition/SIP General Parameters/Fax Signaling Method: 0 = No Fax

Then, set FaxTransportMode = 0 ; This is at Configuration/Media Settings/Fax-Modem-CID Settings/Fax Transport Mode: 0 = Disable = transparent mode

Then, set the transport types:

V21ModemTransportType = 0 ; at Configuration/Media Settings/Fax-Modem-CID Settings/V.21 Modem Transport Type: 0 = Disable = transparent mode

Do the same for V22ModemTransportType = 0 (disable) , V23ModemTransportType = 0 (disable) , V32ModemTransportType = 0 (disable), V34ModemTransportType = 0 (disable)

The docs also say to set BellModemTransportType = 0 , but that is apparently NOT in the web interface.  However, 0 = disable is the default for BellModemTransportType.

Note that those parameters are as shown in the configuration file.  They are described in the table in section 10.9, “General SIP Parameters”. IsFaxUsed is mentioned on page 385. It is ISFAXUSED in the configuration file, and it is under Configuration/Protocol Configuration/Protocol Definition/SIP General Parameters/Fax Signaling Method in the web interface.

At this point, I was able to send and receive faxes with no problem.  If you have trouble, be sure to check out the troubleshooting tips above.

Yealink SIP-T38G Openvpn VPN not functional

Update — the nice people at Voipsupply.com got me a firmware upgrade (38.0.0.70) which is not posted anywhere on the Yealink website.  Although it does cause the phone to upload the vpn configuration file, it still doesn’t work.  Specifically, a Wireshark trace shows absolutely no packets going to the openvpn server.  That is in contrast to the exact same process on the T28 using the same configuration file (I know, but we are testing), which DOES send UDP packets to the openvpn server and correctly set up the vpn and register the phone.

I have sent them the Wireshark traces, config files, and syslogs from the phone.  We will see what they come up with.  But for know, the Openvpn on the T38 is still not functional.

Update — see below – although openvpn does not work on the Yealinlk SIP-T38G, it DOES work on the Yealink SIP-T28P

I was looking for a secure and simple way to provision an IP phone, and came across the Yealink SIP-T28 phone mentioned in the Elastix Asterisk distribution security documentation. Openvpn is easy to configure, and using Openvpn would allow a simple solution for data encryption (control and RTP), as well as firewall traversal. I saw several posts from individuals who had the SIP-T28 Openvpn working (in spite of poor documentation from Yealink). I have purchased a number of phones from Voipsupply.com, and looking at their website, I saw the SIP-T38G, which looked like an update of the T28, with a color screen as well. The docs for the SIP-T38G on the Yealink website, as well as the data sheet for the SIP-T38G on the Voipsupply.com website, said that the T38G had the Openvpn functionality as well, so I ordered a SIP-T38G from Voipsupply.

The other posts helped with the construction of the openvpn configuration file (they said that the hardest part was finding a tool to create a tar file using ‘.’ as the root, but actually the standard tar utility did that very easily, using something like ‘tar cvf ../client.tar .’, putting client.tar in the parent directory to avoid a warning from tar.) However, when I went to upload client.tar to the phone, there was no option to do so!

Below, you can see that the Openvpn functionality is advertised on the box that the phone came in. However, all of the vpn configuration sections are missing from both the web configuration page and the on-phone configuration menus. I have included the figure from the SIP-T38G manual which I downloaded from the Yealink website, as well as a screen capture showing that the configuration options are missing.

I wanted to contact someone from Yealink about this, but there is no usable contact information (other than a call to China, which I don’t consider a viable option). There is a Yealink UK website with a support forum, so I tried to register. I got an immediate email that my registration would be reviewed by their administrator, and would be inactive until it was. Five days later, I have not heard any more from them. I also tried sending an email to Voipsupply support asking them why the phone they sent did not have the capabilities advertised for it in the data sheet on their website (as well as on the box the phone came in), but five days later, I have heard nothing.

Since I bought this phone entirely for the Openvpn capability, what I now have is a very expensive paperweight (albeit one with a beautiful color screen). I posted this to save others the trouble. I have also ordered a T28, which others say they have made work with Openvpn.

Follow-up: I got my Yealink T28 — completely different story there. The VPN entry was on the advanced network configuration screen, right where it was supposed to be. Uploading the client.tar file was a snap (albeit because I had already constructed the client.tar for the T38G!), and configuration was quick and easy. After uploading client.tar and enabling the VPN, I just went to the Account tab on the web interface, entered the extension number under Label, Display Name, Register Name, and User Name, entered the SIP password under Password, and entered the Asterisk machine’s internal tun interface’s IP address under SIP Server, clicked “Confirm”, and I could make phone calls over the VPN!

If you have trouble, confirm that you have the tun interface’s IP address for the SIP Server (not the Asterisk machine’s external IP), that openvpn is started on the server, and that you can ping the Asterisk machine’s internal tun interface IP from the Asterisk machine. You should see the vpn being set up in the /var/log/messages log, along with the IP assigned to the phone’s end of the vpn, and you should be able to ping the tun interface in the phone from the Asterisk machine. And don’t forget to check that the firewall is not getting in the way.

Also, although I did not do this, if you run the openvpn server process on a machine other than the Asterisk machine, you will have to make sure you have the routing entries to get the packets to your Asterisk server. In that case, I would start by making sure that the vpn is set up correctly, and then work on the routing.

Using external SATA / eSATA hard drive with Scientific Atlanta EXPLORER 8300 and 8300HD DVRs from COX

The hard drive that worked was the Toshiba PH3100U-1EXB 1 terabyte external hard drive (available right now at Fry’s Electronics for $99!) The one that did not work was a two terabyte dual drive raid external hard drive of a different brand.

A friend has a Scientific Atlanta EXPLORER 8300 DVR rented from COX Communications in Arizona. It has a fairly small amount of storage (about 74 GB, according to the info – see below), and has an external connector labeled “SATA”. I Googled it, and it looks like some people have had success adding an external hard drive. However, details as to which drives worked and didn’t work are hard to come by. I couldn’t find anything on Scientific Atlanta’s website (apparently now part of Cisco), and when I called COX, the tech said to call Scientific Atlanta myself, since COX doesn’t support adding an external hard drive and didn’t know how to do it (an uncharacteristically poor customer support experience for COX, which has usually had excellent customer support in my experience).

I went to Fry’s electronics, and they said I needed some sort of “media extender”, which they didn’t have. They said that using an eSATA drive was hit and miss. So I missed and then I hit. I wanted to report my experience to save others the trouble.

I am not customer support for either COX or Scientific Atlanta/CISCO. I don’t know if this will work for others, and I don’t know what other combinations might work. I take absolutely no responsibility for any damage you may do by repeating my experience. There is at least some chance that you will lose any recordings you have, so if that is a concern, be sure to watch and/or copy them elsewhere (although I lost no recordings). However, if you find other combinations that work, with this or other DVR’s, please post your findings in the comments below.

First, and most important, check to see whether you have an eSATA or a SATA connector. In my case, it was an eSATA connector labeled “SATA”. Look online for pictures of the difference. The SATA connectors have an “elbow” inside that the eSATA does not have. You can use a digital camera to take a detailed close-up if needed.

Second, you will need an external hard drive with an eSATA connector (most only have USB or perhaps Firewire connectors). You also need to purchase the cable to connect the eSATA connector on your drive to the eSATA connector on the 8300. Different eSATA cables have different connectors, so be sure to get the right one.

To connect the drive, first turn the 8300 off, then unplug it for at least 15 seconds. Be sure both the drive and the 8300 are unplugged, and connect the eSATA cable between them. Leave the drive unplugged, but plug the 8300 back in. It will go through its boot sequence, then the panel on the front of the 8300 will go black. You then have to wait for it to get a signal from the cable company, and the time will then come on the screen. That may take five or ten minutes.

Once the 8300 has the time showing, turn on the 8300 and the TV so you are watching TV through the 8300. Now, plug in the drive, and if needed, turn on the switch so the light on the drive comes on. Now comes the moment of truth.

The first time I did this, I got a message on the TV that the drive or cable were working improperly. I had to press a button on the remote to dismiss the message, and was informed that the external storage will not be working. You should check the cable, but in my case, after several tries, I always got the same message. Unplug everything and disconnect your drive. Fortunately, Fry’s Electronics has a great return policy!

The second time I did this, success! When I turned on the drive, I got a message that the drive would have to be formatted, and I would lose all information on the drive, including any saved recordings. I pressed the button to proceed. There was no indication that anything was happening, and the 8300 continued to work as normal. I left it running for about 40 minutes, to give plenty of time for formatting, although I don’t really know how long formatting takes. After 40 minutes, still no indication of progress, and the 8300 indicated that its storage was as full as it has ever been.

So after 40 minutes, I unplugged the 8300 again for 15 seconds, and plugged it back in to reboot the 8300. After it came back up, I turned it on with the remote. I left for a while, and when I came back the 8300 would only play PBS, and attempts to change the channel or bring up the list of saved programs did not work. I wasn’t sure what was going on, so after a little while I unplugged the 8300 again, disconnected the external drive, and plugged the 8300 back in again. I then saw a message about advanced functions temporarily not working even with the external drive disconnected, and again I could only watch PBS. Therefore, I unplugged the 8300 again, again connected the eSATA cable to the external drive, plugged in the external drive, and plugged in the 8300, and just left it alone for a while. About an hour later, I went back, turned on the 8300 with the remote, and the 8300 was working normally, except that the available space had increased dramatically (95% full went to 6% full)! All of the saved programs were still there, but there was a lot more empty space.

Next time, once the external drive was formatted, I would unplug the 8300 for 15 seconds, leave the external drive plugged in, and then plug the 8300 back in, and leave it alone for an hour.

Update for the 8300HD: I tried the same thing for a 8300HD unit. I unplugged the 8300HD for 15 seconds, connected the external drive (a different specimen of the same model), plugged in only the 8300HD (not the drive) until the time showed on the 8300HD, used the remote to turn on the 8300HD so I was viewing the 8300HD’s signal through the TV, then plugged in the drive and pushed the on button on the drive. The light on the drive came on, and I got the dialog asking me whether to “Format this external storage device to work with this DVR?”. I pressed the “A” key for “Yes, Format”. The dialog disappeared, and there was little or no indication that anything was happening. After about five minutes of nothing, including no flashing of the light on the drive, I again unplugged the 8300HD for 15 seconds, and plugged it back in. After the time showed on the 8300HD, I turned on the 8300HD with the remote, and got a message box that said: “The external storage device connected works with this DVR. NOTE: To safely unplug this device, first unplug power from the DVR, then wait 10 seconds before disconnecting.” After about 10 seconds, that message box disappeared, and the DVR worked normally, but had a lot more empty space! So, on the 8300HD, the whole process only took about 15 minutes.

Update for the Explorer 8240HD The original 8300 was replaced with an 8240HD, for high definition. The hard drive that was used with the 8300 originally was disconnected after unplugging the 8300 for 15 seconds. It was moved to the new 8240HD, and hooked up with the same result as with the 8300HD above. Note that, since the hard drive does not come on until its button is pushed, if there is a power failure, someone will have to push the on button for the hard drive for it to work again. I tested to see what happens if the button is not pushed until after the 8240HD turned on, and the only notable effect was that there was less space available on the 8240HD. When the drive was turned on, I again got the message the the external storage works with this DVR. The 8240HD has a 148 GB disk installed, and I added 931 GB (according to the info page) with the new disk.

The old 8300 appears to be confused about how much space is available now, showing a screen indicating that there is still as much space as there was before. Many of the recorded shows are still in the list, but as expected, trying to play them does not work, but rather the DVR goes right to the “Press the list button to see your recordings” screen.

Reviewing postings on the Internet, I would be careful to be sure that the 8300 is unplugged prior to allowing the external drive to lose power or be disconnected, for fear of data corruption and/or loss of all your recordings. Also, the recordings on the drive are reportedly encrypted with a key that is specific to the serial number of your 8300, so if the drive is moved to another box, you will not be able to view the recordings there.

To see info about your 8300, including info about the attached internal and external drives and what interfaces have been enabled by COX, use the keys on the front of the 8300. Press and hold the “SELECT” button until the little flashing mail icon comes on, then release the “SELECT” button and press the “INFO” button. Page forward and back with the Volume + and Volume – buttons. Press the “EXIT” button when you are done.

If you have success with other DVR’s and/or drives or have other experiences, please let me know in the comments!

Embarcadero C++ Builder – Sending UnicodeString via COM truncates string to half

CB2009 defines a BSTR as a wchar_t*, which is the type returned by c_str() when UNICODE is set. Therefore, if you call a COM function which expects a BSTR with UnicodeString.c_str(), the compiler returns the wchar_t* which the function expects. Sounds good! Except it doesn’t work!!

Actually, both BSTR and the character array in UnicodeString prefix the array of wide chars with a four-byte integer that gives the length. However, BSTR expects this to be the length IN BYTES, whereas UnicodeString makes this the length IN CHARACTERS. Thus, the server gets the wchar_t*, looks right before the pointer for an int, and only uses that many bytes. So, if your UnicodeString is seven chars long, and thus 14 bytes long, CB2009 sets the length to seven, and COM only accepts the first seven bytes (and thus the first four chars). So, all your strings are cut in half!

To make things worse, UnicodeString.Length() does not count out the length to the null terminating char, but rather just returns the integer. So, if you fix the integer to be 14 as COM expects, UnicodeString.Length now returns 14 for your seven character string! We dare not mess with UnicodeString’s data.

The solution is to use WideString instead. Instead of:

UnicodeString myString = L”My Data”;
ptr->ComFunctionExpectingBSTR(myString.c_str()); // this compiles, but COM only uses the first half of the string

use

UnicodeString myString = L”My Data”;
ptr->ComFunctionExpectingBSTR(WideString(myString).c_bstr()); // this has the correct length

References:
http://msdn.microsoft.com/en-us/library/ms221069.aspx – Microsoft’s reference for the BSTR type in MSDN/Win32 and COM Development/Component Development/COM/Automation Programming Reference/Data Types, Structures and Enumerations/IDispatch Data Types and Structures

http://docwiki.embarcadero.com/RADStudio/en/Unicode_in_RAD_Studio#New_String_Type:_UnicodeString – Unicode in RAD Studio

NOTE: In my testing, this is a problem sending data to Microsoft Outlook 2007. It did not appear to cause a problem when sending to Crystal Reports, so the issue might well be with the server code rather than the COM subsystem, depending on how the server determines the length of the passed string. I have seen example code that just passes a WideString to COM, but for me, the compiler gives a Type mismatch error unless I pass the pointer returned by c_bstr()

To see how the length of the string is set:

wchar_t* lit = L"My Sttring";
int* litIPtr = (int*) lit;
litIPtr--;
int litLen = *litIPtr;

WideString ws = lit;
wchar_t* wsPtr = ws.c_bstr();
int* wsIPtr = (int*) wsPtr;
wsIPtr--;
int wsLen = *wsIPtr;

UnicodeString us = lit;
wchar_t* usPtr = us.c_str();
int* usIPtr = (int*) usPtr;
usIPtr--;
int usLen = *usIPtr;

// int actualLen = StrLen(lit); // if UNICODE is set
int actualLen = wcslen(lit);

ShowMessage("For a " + String(actualLen) + " character string, Literal gives " +
String(litLen) + ", WideString gives " +
String(wsLen) + ", UnicodeString gives " + String(usLen));

Windows Communication Foundation 3.5 Unleashed Errors

I am reading this book; it is very well written, and the people who wrote it are obviously very smart. Unfortunately, it is chock full of technical errors. I have wasted hours trying to get the examples to work. In the hope of saving others some time, I am listing a few of the problems I found.

In addition to these, there are lots of other errors I found in my first, casual, reading, especially in the code.

In the Fundamentals chapter (2)

Page 45, step 1: add the app.config file to the Host project, NOT the DerivativesCalculatorService project (see page 53)
error: Service ‘DerivativesCalculator.DerivativesCalculatorServiceType’ has zero application (non-infrastructure) endpoints. This might be because no configuration file was found for your application, or because no service element matching the service name could be found in the configuration file, or because no endpoints were defined in the service element.

Page 56, step 4: there should not be a colon before “svcutil” (and there is not in the screenshot below)

Page 58: The class should be DerivativesCalculatorClient rather than DerivativesCalculatorProxy (since that is the class created by the tool — see listing 2.6)

Page 67 – 72: Getting the service to run under IIS 7.0 required quite a number of additional steps, at least in my configuration.

1) — Page 69, Step 3 – Must use Add Application rather than Add Virtual Directory, at least in IIS 7.0 . I wasted quite a bit of time on this one, until I found http://social.msdn.microsoft.com/forums/en-US/wcf/thread/49d9279f-2bc1-482b-8bb0-da1261736acb/ , where this exact problem with this exact same example was noted in April 2006.
error: The type ‘DerivativesCalculatorService.Calculator’, provided as the Service attribute value in the ServiceHost directive could not be found.

2) — Need to give the IIS_IUSRS permission to the DerivitivesCalculatorService directory, so IIS can use the config file

3) — Need to give the Anonamous Login user permission to the DeriviativesCalculatorService directory, so IIS will serve it to the user

4) — I had to fix the bindings to the .svc extension using
c:windowsMicrosoft.NETFrameworkv3.0Windows Communication FoundationServiceModelReg -r
It is possible that I had that problem because I enabled IIS after I had already installed VisualStudio. Or not.

5) — I had to turn on the WCF Activation with Control Panel/Programs/Turn Windows Features On/Microsoft .NET 3.0/WCF Activation for http and non-http – see http://michael-arnett.spaces.live.com/blog/cns!5AA848FF3F707F99!1093.entry?
– error: HTTP 500 – Handler svc-Integrated has a bad module “ManagedPipelineHandler” in its module list

Page 75 – in my testing, the MSFT string was NOT encrypted when I used the netTcpBinding

Page 79 – to use the WcfTestClient with this demo, run:
WcfTestClient http://localhost:8000/Derivatives/
or
WcfTestClient http://localhost:8000/Derivatives/?wsdl

Neopost IJ25 Postage Meter Pricing Scam

I just got off the phone with Neopost, whose postage meter I have been using for a number of years. I guess I never put together the true cost of using it. The reason for my call was that my postage meter suddenly stopped working, saying “Warning, Ink Expired”.

It turns out that the Neopost IJ25 postage meter is programmed to stop working if the same ink cartridge has been installed for more than one year (as in 365 days). When I called, I was told that we should have received a single warning about a month ago that the ink would expire soon. However, if we had pushed “Ok” that one time, the warning would disappear, and the next warning would be when the postage meter stopped working altogether. I don’t know if we got the warning or not, although nobody remembers seeing it.

Nonetheless, my Neopost IJ25 postage meter is dead in the water. The cost for a replacement ink cartridge is $124: $86 for the cartridge (!!), $29 for overnight shipping “& handling” (!!), and $9.32 for tax. 2-3 day shipping would have been $23, 10 day shipping $12. Clearly, Neopost has a strong incentive to set up their IJ25 postage meter so we would not see the warning, and is not shy about overcharging for shipping.

In addition, Neopost gouges us to update the meter for postal rate changes (so it prints 43 cents instead of 42 cents) – this was $91.89 on 12/22/08, plus a special deal $67.02 for one year of subsequent updates. In the notice, Neopost gleefully noted that the postal service has agreed to start increasing the postage more often – probably twice a year. The thing is, to get the special deal, I have agreed to automatic, non-cancelable upgrades; to cancel, I must give at least 30 days, but no more than 60 days, notice. Again, clearly designed to make it more difficult to cancel.

The only thing that is reasonably priced is the actual rental of the Neopost IJ25 postage meter and scale, at $219.57 for 12 months.

However, the actual cost of using the Neopost IJ25 meter for 12 months is: $219 rental, about $90 reprogramming, and $124 for the timed ink cartridge, or $433, or about twice the quoted rental cost. Not to mention the inconvenience of having a postage meter that dies unexpectedly and with no (practical) notice.

So why is that a scam? For the same reason that it is a scam when an airline quotes an airfare that doesn’t include baggage. Neopost should be honest about the actual cost of renting their equipment, and not gouge their customers for the rate reprogramming and ink cartridges. There is no excuse for charging $29 for overnight shipment of a 4 ounce cartridge. And, worst of all, they should not design the Neopost IJ25 postage meter to fail unexpectedly and without (practical) warning, significantly inconveniencing their customer.

You can bet that between now and next December, I will be looking into other options that don’t include a meter programmed to fail.

If anyone from Neopost cares to dispute or comment on this post, I will be happy to talk with them. If anything is incorrect, I will certainly correct it.