Jul 25

How to: Get the Sum of the Values from List in C#.Net?

How to: Get the Sum of the Values from List in C#.Net?

Currently I am working on a new project in which I need to calculate some statistics that involve some basic statistics which require the total amount. The easy way out is just to do a foreach statement and iterate through the list adding the values. But somehow, I wanted something that looked fancier but really cleaner and much easier to maintain and understand. Because of that, I came across the Sum method found in a list. In my case, because I was obtaining all the data from an API and it was returned in a Json… it came as a string. So adding the complexity of adding a list of object’s string properties.


In order to achieve the end result: “Summing all string representations of numbers stored as properties in a list of objects in a clean, easy to manage way” we rely on LINQ:

decimal total = myListOfValues.Sum(x => decimal.Parse(x.NumberAsString));

and just like that, we’ve got our numbers all summed up in just one simple line!

Jul 25

.Net: Supported Console Colors

.Net: Supported Console Colors

The days of writing applications that write to the console seem to be over for a while now. Notwithstanding, we find ourselves using the console from time to time to help us debug or quickly test application logic. In my case, I am currently using the console to monitor logging information from my application. It is terribly useful to be able to see progress as it happens. It sort of reminds me of using tail -f in Ubuntu against a log file. As part of my effort to make this log reading more easy I decided I wanted to color code the messages based on what they were: Errors / Warnings / Info / Debug, etc. I kept asking myself if Windows is able to support colored console as Ubuntu does, and I am happy to report it does! So here is when the question arises:

What colors are supported and how to use them?

Thankfully there is an enumeration to help us answer this question:

ConsoleColor

Member name Description
Black

The color black.

Blue

The color blue.

Cyan

The color cyan (blue-green).

DarkBlue

The color dark blue.

DarkCyan

The color dark cyan (dark blue-green).

DarkGray

The color dark gray.

DarkGreen

The color dark green.

DarkMagenta

The color dark magenta (dark purplish-red).

DarkRed

The color dark red.

DarkYellow

The color dark yellow (ochre).

Gray

The color gray.

Green

The color green.

Magenta

The color magenta (purplish-red).

Red

The color red.

White

The color white.

Yellow

The color yellow.

Jul 25

Pro Tools: Json

Pro Tools: Json

I’ve recently started working on a small programming project on the site. It has been a while since I’ve done any programming, so seeing all the new technologies and cloud offerings I must confess I am as excited and fascinated as I am daunted by how much there is and how far it has evolved. This brings us to today’s post: “Pro Tools: Json.” As you probably know Json is used widely nowadays to exchange information over API calls. Seems the days of XMLs are in the past, and although I ignore the reasons I do welcome Json (mostly because I have no choice I guess, but still.) So as I am welcomed into this new world full of Json (I had already used it extensively in configuration files for different applications out there) I figured I would start a collection of different tools I have found that have helped a lot. Hope this helps others as well!


Json2CSharp.com

I am not sure how it is with other programming languages, but in C# you’ve just got to serialize and deserialize your objects. As you work with any dataset there is nothing quite like representing the data in your familiar object data structure. Because of this, you need a tool that can help you get the classes you require to be able to deserialize the json into something you can manage with your application’s logic. Here comes Json2CSharp. As the name implies, you just provide it some json and it will generate for you the classes. Makes life much simper specially when you are starting or are having a hard time reading the json. A must have tool!

 

Jul 23

Pro Tip: Never use type double for money / financial applications

Pro Tip: Never use type double for money / financial applications

When you are new into coding financial applications one of the mistakes people tend to do is using a data type that it’s precision / significant digits are not enough to properly represent numers. If you’ve taken some CS classes you know how decimals are represented in a computer system; They are fractions.

To avoid this common pitfall, there are a variety of techniques. One for example, is that if you need only 4 positions after the decimal place you simply store them as integers on the backend and dividing them by a power of 10 when presenting them. Some languages offer support of this behind scenes. Take for example System.Decimal in .Net; It is still a floating point but instead of being a floating binary point it is a floating decimal point, and that makes all the difference.

If you don’t believe me, try this code to see how a double won’t work for a financial/money application:

using System;

class Test
{
    static void Main()
    {
        double x = 0.1;
        double y = x + x + x;
        Console.WriteLine(y == 0.3); // Prints False
    }
}

So if you are using .Net, you should consider using System.Decimal since decimal uses a scaling factor of multiples of 10, numbers like 0.1 can be represented exactly. In essence, the decimal type represents this as 1 / 10 ^ 1, whereas a double would represent this kind of like 104857 / 2 ^ 20.

A decimal can exactly represent any base 10 value with up to 28/29 significant digits (like 0.1).

References:

decimal (C# Reference) https://msdn.microsoft.com/en-us/library/364x0z75.aspx

decimal (C# Reference)

Visual Studio 2015
  The decimal keyword indicates a 128-bit data type. Compared to floating-point types, the decimal type has more precision and a smaller range, which makes it appropriate for financial and monetary calculations. The approximate range and precision for the decimal type are shown in the following table.
Type Approximate Range Precision .NET Framework type
decimal (-7.9 x 1028 to 7.9 x 1028) / (100 to 28) 28-29 significant digits System.Decimal

If you want a numeric real literal to be treated as decimal, use the suffix m or M, for example:

decimal myMoney = 300.5m;

Without the suffix m, the number is treated as a double and generates a compiler error.

The integral types are implicitly converted to decimal and the result evaluates to decimal. Therefore you can initialize a decimal variable using an integer literal, without the suffix, as follows:

 decimal myMoney = 300;

There is no implicit conversion between floating-point types and the decimal type; therefore, a cast must be used to convert between these two types. For example:

decimal myMoney = 99.9m;
double x = (double)myMoney;
myMoney = (decimal)x;

You can also mix decimal and numeric integral types in the same expression. However, mixing decimal and floating-point types without a cast causes a compilation error.

For more information about implicit numeric conversions, see Implicit Numeric Conversions Table (C# Reference).

For more information about explicit numeric conversions, see Explicit Numeric Conversions Table (C# Reference).

Formatting Decimal Output

You can format the results by using the String.Format method, or through the Console.Write method, which calls String.Format(). The currency format is specified by using the standard currency format string “C” or “c,” as shown in the second example later in this article. For more information about the String.Format method, see String.Format.

Jul 10

How to: Install a valid SSL Certificate for Ubiquiti Networks’ Unifi Controller

How to: Install a valid SSL Certificate for Ubiquiti Networks’ Unifi Controller

Problem

If you are like me, you are probably sick by now of having the certificate error page pop up everytime you visit the controller’s page. I personally think Ubiquiti should make it as easily as uploading a web certificate via the GUI / Web page but for now I am just thankful there is a way to do via the command line. You should note I have only been able to install a web certificate on the Unifi Controller (The one that controls networking equipment) and the Unifi Voip Controller. The Mfi controller and Video controller seem a bit behind the times and I have not been able to assign them proper web certificates yet.

Solution

UniFi relies on HTTPS for extra security. This means that the browser will check for valid certificates when making a secure connection to the web server. Although the alert message may prove annoying, there’s no risk to the connecting user. To avoid this error:

  1. Buy a signed SSL certificate from any web hosting provider.
  2. Then make the following changes to the controller:
    sudo su -
    # cd <unifi_base> 
    # on Windows, "%USERPROFILE%/Ubiquiti Unifi"
    cd /usr/lib/unifi 
    
    # create new certificate (with csr)
    java -jar lib/ace.jar new_cert <hostname> <company> <city> <state> <country>
    
    # your CSR can be found at /var/lib/unifi
    # - unifi_certificate.csr.der
    # - unifi_certificate.csr.pem
    
    # have this CSR signed by a CA, you'll get a few certificates back...
    # copy the signed certificate(s) to <unifi_base>
    
    # import the signed certificate and other intermediate certificates
    java -jar lib/ace.jar import_cert <signed_cert> [<other_intermediate_root_certs>...]

     

I have used this instructions (obtained from: https://help.ubnt.com/hc/en-us/articles/212500127-UniFi-SSL-certificate-error-upon-opening-controller-page) successfully. Remember the unifi base folder changes depending on what unifi controller you are using (video, voip, mfi, base, etc.) The instructions imply only the base controller has this fuctionality but I was able to do it with the voip controller as well.

Jul 10

How to: Assign multiple IP addresses to one interface in Ubuntu using the Command Line Interface (CLI)?

How to: Assign multiple IP addresses to one interface in Ubuntu using the Command Line Interface (CLI)?

I have been working with Ubuntu more lately and ran into the need to direct traffic going to one server (via IP) to go to a new server but I couldn’t change the client’s configuration. Because part of the services were already migrated over I could not change the IP address to the old server. Because of this I required a way for the new server to answer on the new IP address assigned to it but also listen to the old server’s IP address. To do this you need your server to have more than one IP address which is possible (have done it with Windows Servers for several years) so just needed to find out how to do it on Ubuntu


Solution

  1. If you need an additional IP address just for the moment you can add it to any interface on your machine with
     sudo ip address add <ip-address>/<netmask> dev <interface>
    

    for example

     sudo ip address add 172.16.100.17/24 dev eth0
    

    would add 172.16.100.17 using a 24bit netmask to the list of addresses configured for your eth0.

    You can check the result with

    ip address show eth0
    

    and you can delete this address again with

    sudo ip address del 172.16.100.17/24 dev eth0
    

    Of course these changes are lost when you reboot your machine.

  2. To make the additional addresses permanent you can edit the file /etc/network/interfaces by adding as many stanzas as needed of the form:
    iface eth0 static
        address 172.16.100.17/24
    

    so if you needed say 2 IP addresses you would do so as follows:

    iface eth0 inet dhcp
    
    iface eth0 inet static
        address 172.16.100.17/24
    
    iface eth0 inet static
        address 172.16.24.15/24
    

    Note we left the dhcp on the first line, so you will end with 3 ip addresses, 2 static ones provided above and one assigned to you by the DHCP server which will be your primary address.

    To activate these settings without a reboot use ifdown/ifup likeso:

    sudo ifdown eth0 && sudo ifup eth0
    

    It is essential to put those two commands into one line if you are remoting into the server because the first one will drop your connection! Given in this way the ssh-session will survive and you’ll be able to reconnect. Exercise caution as if there is an error networking will be dropped but because of the config error it won’t come back up.

 

Jul 10

Resolved: DNS Lookup Order with VPN on OSX

Resolved: DNS Lookup Order with VPN on OSX

Question: Why can’t I access intranet resources over my VPN connection?

After performing an update on my Mac I wasn’t able to access intranet resources over my VPN connection. Further troubleshooting revealed that via IP address I could access servers but the DNS resolution wasn’t working.

Answer: Because you are not using your intranet’s DNS Server

Many people are having issues when connecting to the VPN because the DNS preference of your local connection vs your remote one. However, the solution to his problem is quite simple and it is almost given by the question itself (when talking about connection preference.)

There are several workarounds, but the easy solution is that you need to give your VPN connection a higher order in the network settings so that the VPN DNS Server is queried before the main internet connection’s DNS is.

To do this simply follow this instructions:

  1. Go to the Network section in the System Preferences.
  2. On the bottom use the gear icon and select the (probably fourth) option that reads roughly “Establish the order of the services” (I say roughly because my system is in Spanish and I don’t have one in English to see the exact text)
  3. There, simply drag and drop your VPN connection above your internet connections; or for that matter above it all so it’s number 1. This will make the VPN’s DNS Server the preferred choice helping you resolve DNS names for your intranet.

and you’re done!

I am not sure why, but probably during an update or something the “service order” in “Network Preferences” moved around a little and when I originally set up my VPN connection it was in the correct order for me so I never realized this was a feature/option. Took a while to find but glad it wasn’t something overly complex.

Mar 27

Resolved: How to recover an accidentally deleted volume (partition) in a Virtual Disk protected by BitLocker

Resolved: How to recover an accidentally deleted volume (partition) in a Virtual Disk protected by BitLocker

I am going to have to start with a confession, “Resolved” might be a bit of a stretch. For starters, the best case scenario looks like recovering the information but you’ll need a temporary place where to store it, test it (make sure it’s fine not corrupted), etc. I can’t stress enough that as soon as you have messed something up you should read everywhere all you can to avoid following one method and screwing your chances of real recovery with it. I say this because if you write on top of your old data it’s gone probably for good. You have to be extra careful and be mindful that one method might mess up your chances with another one, so err on the side of caution. By this point all bets are off, you should have had made a backup and this should not be your only alternative. If the information is valuable you’re better off hiring a professional than trying to fix it yourself. So all I can really say now is I take no responsibility, this method seemed to have worked for others and it worked for me thankfully… it does not mean it will work for you. So now that you understand the risks involved (if not please abort), let’s get on with it!

So as you probably figured out, I deleted a volume (partition/drive) accidentally. I was trying to delete one that had an issue so I backed up my data on that volume, then moved it over to another, and then I deleted it so I could recreate elsewhere… oh wait… it’s still there… oh wait… where is my other data volume… oh sh!t! And that is how I lost my entire long easter weekend. I tried several utilities on the internet (albeit I distrust so many third parties on my server but what choice did I have), and no good. Finally I found an article from Microsoft: How To Recover an Accidentally Deleted NTFS or FAT32 Dynamic Volume. Basically it states:

To Recover a Deleted NTFS Volume

  1. Re-create the exact same volume but choose not to format it. This may be difficult if you do not remember the exact size you had created originally, especially because the Disk Management snap-in tends to round partition sizes.
  2. Using Dskprobe.exe, recover the backup boot sector for the NTFS volume from the end of the volume. Because it is a dynamic volume you may need to use Dmdiag.exe to help find the backup boot sector, or search for it by using Dskprobe.exe (on the Tools menu, click Search Sectors).
  3. After rewriting the NTFS boot sector, quit Dskprobe.
  4. In Disk Management, click Rescan Disks on the Action menu. This should mount the volume for immediate use.

so I did that the best I could. Fortunately my partition covered the whole drive so I just used the max size while creating it. Please not you should NOT FORMAT the partition/volume, if you do then you just made the recovery considerably less likely and harder and out of the scope of this post. By now the partition showed up as RAW (not formatted) and all the recovery utilities where unable to recover the information I needed. By now I had lost almost all hope until I started to look at the actual data looking for any information on any sector when I realized everything looked like garbage… eh I mean, random text that might as well be encrypted data! So here is where things took a turn and my immediate future looked brighter! The deleted volume was encrypted using BitLocker, and because of that the OS was unable to recognize the formatting on the drive and what not. Doing some research I found a system utility designed for scenarios such as this.


Solution

Let’s just get a quick list of pre-requisites before we get started

  1. Make sure you have NOT done a thing to your deleted volume/partition… formatting specially.
  2. Find your decryption key for BitLocker. You get a few choices between:
    1. Recovery Key
    2. Recovery Password
    3. Password
    4. Key Package
  3. Find a suitable storage location to store your decrypted data
    1. The volume/partition must not be the one you are trying to recover
    2. The volume/partition must have at least the same amount of available space as the one you are trying to recover (total size of both include used and free space)

so now that you have everything you need let’s get started!

First of, take a deep breath and make sure you don’t accidentally make this thing worse. Double, Triple check everything to make sure you are doing the right thing. So let’s get started


Step I – Recreate the partition from Disk Management

I’ll start off with a big warning: Do NOT format the volume/partition. There are several ways to launch Disk Management (compmgmt.msc), so go ahead and use your favorite method and launch it with administrative rights (as you’ll need them to create the partition.) Now that you have it open, time to go look for the Unallocated space where you had the partition you are trying to restore.

Once identified, let’s proceed to recreate the partition. In my case, I right clicked the Unallocated space and created a New Simple Volume. Keep in mind we are trying to recreate the lost partition, so you need to provide all the information identically as it was on the deleted volume (size, etc.) For more advanced scenarios you could specify the start and end sectors but if you are like me and used the entire disk the wizard should be enough. Again, be careful, one of the steps in the wizard reads “Format Partition.” Make sure you select the option “Do not format this volume” to avoid data loss. Don’t forget to mount it to a drive letter so you can work on it later on (in my case I have assigned it the letter E.) Once you’re done you’ll see in the Disk Management console that your partition appears as RAW. If you didn’t have BitLocker on your partition it should (based on what I read) show your original partition and you should had recover access to your data. In our case because we used BitLocker the information is encrypted and the OS does not recognize it as a BitLocker enabled drive. Here is where Step II comes in


Step II – Use repair-bde to unencrypt your BitLocker volume

Due to Microsoft wanting to be careful and safe with your data, this tool is basically a read-only tool. It will not repair your lost volume, but rather would read it, decrypt it, and save the unencrypted information elsewhere. This is why we need a storage location where we can store the entire content of the encrypted volume (not just the used space, but the entire space of the volume.) To do so, we are going to use the repair-bde tool as follows:

repair-bde <InputVolume> <OutputVolumeorImage> [-rk] [–rp] [-pw] [–kp] [–lf] [-f] [{-?|/?}]

where

Parameter Description
<InputVolume> Identifies the drive letter of the BitLocker-encrypted drive that you want to repair. The drive letter must include a colon; for example: C:.
<OutputVolumeorImage> Identifies the drive on which to store the content of the repaired drive. All information on the output drive will be overwritten.
-rk Identifies the location of the recovery key that should be used to unlock the volume. This command may also be specified as -recoverykey.
-rp Identifies the numerical recovery password that should be used to unlock the volume. This command may also be specified as -recoverypassword.
-pw Identifies the password that should be used to unlock the volume. This command may also be specified as -password
-kp Identifies the recovery key package that can be used to unlock the volume. This command may also be specified as -keypackage.
-lf Specifies the path to the file that will store Repair-bde error, warning, and information messages. This command may also be specified as -logfile.
-f Forces a volume to be dismounted even if it cannot be locked. This command may also be specified as -force.
-? or /? Displays Help at the command prompt.

In my case, I used the following command first:

.\repair-bde.exe E: N:\RecoveredData.img -rp 123456-789012-345678-901234-567890-123456-789012-345678 –lf C:\log.txt

This is what I’ve seen most people use. I haven’t troubleshoot enough but I did ran into some issues using an Imagine file. I tried to mount it with no luck (The disk image file is corrupted.) and people suggested using 7Zip to open the image file. That worked fine but all the information I got out was corrupted. I did experience an issue where the progress got stuck at 17% and I had to click enter to have it continue progressing. The same thing happened at different progress %s which might had been the root cause of the data corruption.

Because of those issues, I prepared a new volume where to store the data. Instead of using an image file I used a volume instead like so:

.\repair-bde.exe E: B: -rp 123456-789012-345678-901234-567890-123456-789012-345678 –lf C:\log.txt

If everything goes well, you should get a message like this one:

BitLocker Drive Encryption: Repair Tool version 6.3.9600

Copyright (C) 2013 Microsoft Corporation. All rights reserved.

Beginning scan for BitLocker metadata.

Scanning boot sectors for pointer to metadata: 100%

Scanning sector boundaries for metadata: 100%

Finished scanning for BitLocker metadata.

Beginning decryption.

Decrypting: 100% Complete.

Finished decryption.

ACTION REQUIRED: Run 'chkdsk B: /f' before viewing decrypted data.

after that go ahead and run chkdsk B: /f as requested. (note I used B you might have used another drive letter)

The type of the file system is NTFS.

Chkdsk cannot run because the volume is in use by another

process.  Chkdsk may run if this volume is dismounted first.

ALL OPENED HANDLES TO THIS VOLUME WOULD THEN BE INVALID.

Would you like to force a dismount on this volume? (Y/N) y

Volume dismounted.  All opened handles to this volume are now invalid.

Volume label is NTFS Store.

Stage 1: Examining basic file system structure ...

  522240 file records processed.

File verification completed.

  1912 large file records processed.

  0 bad file records processed.

Stage 2: Examining file name linkage ...

  591156 index entries processed.

Index verification completed.

  0 unindexed files scanned.

  0 unindexed files recovered.

Stage 3: Examining security descriptors ...

Security descriptor verification completed.

  34459 data files processed.

CHKDSK is verifying Usn Journal...

  545242136 USN bytes processed.

Usn Journal verification completed.

Correcting errors in the uppercase file.

CHKDSK discovered free space marked as allocated in the volume bitmap.

Windows has made corrections to the file system.

No further action is required.

   3145597 MB total disk space.

662182044 KB in 485658 files.

    155952 KB in 34460 indexes.

         0 KB in bad sectors.

   1223991 KB in use by the system.

     65536 KB occupied by the log file.

   2497588 MB available on disk.

      4096 bytes in each allocation unit.

805273087 total allocation units on disk.

639382591 allocation units available on disk.

and voilà! You got yourself a volume (B:) with the information that used to be stored in the encrypted drive!

I am still double checking all my data is there and that no information is corrupt but thus far it is incredibly promising. All the files I have tried to access are there and displaying properly. Until the users come in tomorrow morning I won’t know for sure if this was 100% successful but from what I’ve seen I believe so. Hopefully this guide saves your life data as well!


Additional knowledge

So here are a few bits and pieces of additional knowledge that might help you when facing this issue:

Q: I get an error when running repair-bde: “Failed to authenticate using supplied recovery information. (0x80310000)”

A: You are not providing the right key/password to decrypt your BitLocker drive. As the message at the end says “ERROR: BitLocker is not suspended on this volume. Try another key protector.

Q: The image file shows up as corrupted. Could not open it with 7Zip either

A: If your drive is over 2 TB in size it most likely is GPT formatted. If you use diskpart you’ll see that such a formatted disk comes with a “Reserved” system partition:

DISKPART> list partition

Partition ### Type Size Offset
------------- ---------------- ------- -------
Partition 1 Reserved 128 MB 17 KB
Partition 2 Primary 3071 GB 129 MB

In my case, when I deleted my partition, partition 1 of 128mb remained there so I had no issues. You should use diskpart to see if that is the case. If you don’t have it then partition alignment won’t be the same without it. A potential solution would be to grab a similar drive, partition it, and recreate the partitions identically to replicate as they were before you deleted them in the troubled drive. I am no expert on this, so it’s up to you to pick the right tool and not overwrite your data.

Q: This imagine file looks familiar, can I mount it on a Mac?

A: Sure can. Should work with windows as well… not sure what to make it of though.

Mar 23

Resolved: “(initramfs) unable to find a medium containing a live file system” while installing Ubuntu from a USB device

Resolved: “(initramfs) unable to find a medium containing a live file system” while installing Ubuntu from a USB device

Like I mentioned in a previous post I am working on installing an Ubuntu machine in order to do some GPU mining (this time around Ethereum) and now I am running into more issues. When booting up from the USB in order to perform the installation I am getting the following error message:

Resolved: “(initramfs) unable to find a medium containing a live file system” while installing Ubuntu from a USB device

after much digging it seems to be related to the USB issue referred to in Resolved: USB keyboard stops working on Ubuntu installation at language selection.

Solution

The solution is to use USB 3.0 ports only. So what I had to do is sacrifice the mouse (I only have 2 USB 3.0 ports available) and proceeded to have the keyboard and USB drive connected so I could proceed with the installation. To my surprise that did the trick!

Mar 19

Resolved: USB keyboard stops working on Ubuntu installation at language selection

Resolved: USB keyboard stops working on Ubuntu installation at language selection

I am currently working on setting up an ethereum mining rig and I wanted to use Ubuntu as my operating system (no need to pay for a windows license.) However, as I was installing Ubuntu 14.0.4 LTS on my new computer I ran into a curious detail. As soon as I started the installation process my keyboard and mouse stopped working. I was able to go through the initial screen where you select on the boot options that you wish to install Ubuntu. However, as soon as the installation started I lost both my mouse and keyboard on the language selection screen. This was so weird I try to reboot several times until I started reading the error messages and realized it was an issue with the USB controller (by then I had unplugged a lot of peripherals and tried different video cards.)

Solution

As I pointed out the issue was with the USB controller. After reading on the internet this is what I was able to gather. This issue happens on newer hardware, particularly the one that comes with USB 3.0. In order for the USB devices (mouse and keyboard) to work you need to make sure you do the following:

  1. Have enabled legacy mode for the USB controller,
  2. enable USB 3.0 devices,
  3. and here is the key: Connect your Mouse and Keyboard on USB 3.0 ports

and that’s it! Some people have reported simply unplugging the devices and connecting them to different ports do the trick; I am guessing the connected them to USB 3.0 ports afterwards. I tried to change the ports and that did not work in my case. I went from a USB 2.0 port to another 2.0 port just to have the light in the mouse turn on initially and then it went off. Others report just enabling “USB Device Legacy Support” would do the trick. In my case that came enabled right off the bat on the BIOS so that wasn’t enough. It was until I read further and realized there was a conflict with new controllers and that you needed to stick to USB 3.0 ports that I got it to work, hence my recommendation and solution to enable legacy mode and use USB 3.0 ports. Hope that helps!

 

Load more