Java, Strings and Custom Comparators

I recently needed to sort a list of strings according to a custom, non-native order defined at run time. For example given a list of strings:

  1. seven
  2. one
  3. four
  4. three
  5. two
  6. six
  7. 5
  8. N|ne
  9. eight
  10. zten
  11. aleven
  12. btwelve
  13. dthirteen
  14. cfourteen

one might need to sort these dynamically at run time in perhaps the following order, which may or may not change during runtime:

  1. One
  2. Two
  3. three
  4. four
  5. 5
  6. six
  7. seven
  8. eight
  9. n|ne

In addition, the algorithm would need to be able to handle strings that were not defined in the  ’desiredOrder’ list such that unrecognized strings would be placed at the end of the known, ordered list, and then sorted natively from there. An example of how the original list above should look after the sort is:

  1. one
  2. two
  3. three
  4. four
  5. 5
  6. six
  7. seven
  8. eight
  9. N|ne
  10. aleven
  11. btwelve
  12. cfourteen
  13. dthirteen
  14. zten

I decided that creating a new class implementing the Java Comparator interface would be the most efficient way of doing this and was hoping that there would be some immediate cut & paste code gleaned from a simple Google search, but, no dice. Hopefully, at some point in the future, someone else in my position will find this code useful. Here’s what I came up with (comments, suggestions, criticism is welcome):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
public class ADynamicStringComparator implements Comparator
{
    private ArrayList order = new ArrayList();
 
    /**
     * String Comparator class that takes an explicit desired order. 
     * Case insensitive. If an encountered String is not contained within the
     * provided desiredOrder, then it will be placed at the end of the known
     * list within desiredOrder and then sorted natively.  
     * @param desiredOrder 
     */
    ADynamicStringComparator(List desiredOrder)
    {
        if(desiredOrder == null){
            throw new IllegalArgumentException("desiredOrder cannot be null.");
        }
        //set everything to lowercase
        for (String string : desiredOrder) {
            this.order.add(string.toLowerCase());
        }
    }
 
    @Override
    public int compare(String s1, String s2) 
    {
        s1 = s1.toLowerCase();
        s2 = s2.toLowerCase();
 
        //get index for 1
        int indexOfS1 = order.indexOf(s1);
        if(indexOfS1 == -1){
            indexOfS1 = order.size();
        }
 
        //get index for 2
        int indexOfS2 = order.indexOf(s2);
        if(indexOfS2 == -1){
            indexOfS2 = order.size();
        }
 
        if(indexOfS1 == order.size() && indexOfS2 == order.size())
        {
            return s1.compareTo(s2);
        }else{
            return indexOfS1 - indexOfS2;
        }
    }
}

Because the Java List interface preserves order, we can use a list as a parameter to the comparator’s constructor to define our order at runtime. Below is some simple test code as an example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
public void testCompare() 
{
    ArrayList list = new ArrayList();
    list.add("seven");
    list.add("one");
    list.add("four");
    list.add("three");
    list.add("two");
    list.add("six");
    list.add("5");
    list.add("N|ne");
    list.add("eight");
    list.add("zten");
    list.add("aleven");
    list.add("btwelve");
    list.add("dthirteen");
    list.add("cfourteen");
 
    /*The desired sort order. */
    ArrayList desiredOrder = new ArrayList();
    desiredOrder.add("One");
    desiredOrder.add("Two");
    desiredOrder.add("three");
    desiredOrder.add("four");
    desiredOrder.add("5");
    desiredOrder.add("six");
    desiredOrder.add("seven");
    desiredOrder.add("eight");
    desiredOrder.add("n|ne");
 
    /*Another desired sort order. */
    ArrayList desiredOrder2 = new ArrayList();
    desiredOrder.add("Two");
    desiredOrder.add("three");
    desiredOrder.add("four");
    desiredOrder.add("six");
    desiredOrder.add("One");
    list.add("dthirteen");        
    desiredOrder.add("seven");
    desiredOrder.add("eight");
    desiredOrder.add("5");
    desiredOrder.add("n|ne");        
 
    System.out.println("----------------");
    System.out.println("before sort 1");
    System.out.println("----------------");
    for (String string : list) {
        System.out.println(string);
    }        
 
    System.out.println("----------------");
    System.out.println("desiredOrder 1");
    System.out.println("----------------");
    for (String string : desiredOrder) {
        System.out.println(string);
    }
 
    Collections.sort(list, new ADynamicStringComparator(desiredOrder));
    System.out.println("----------------");
    System.out.println("after sort 1");
    System.out.println("----------------");
    for (String string : list) {
        System.out.println(string);
    }
 
    assertEquals(list.get(0), "one");
    assertEquals(list.get(1), "two");
    assertEquals(list.get(2), "three");
    assertEquals(list.get(3), "four");
    assertEquals(list.get(4), "5");
    assertEquals(list.get(5), "six");
    assertEquals(list.get(6), "seven");
    assertEquals(list.get(7), "eight");
    assertEquals(list.get(8), "N|ne");
    assertEquals(list.get(9), "aleven");
    assertEquals(list.get(10), "btwelve");
    assertEquals(list.get(11), "cfourteen");
 
    Collections.sort(list, new ADynamicStringComparator(desiredOrder2));
    System.out.println("----------------");
    System.out.println("after sort 2");
    System.out.println("----------------");
    for (String string : list) {
        System.out.println(string);
    }        
 
    /*Sorry, got lazy writing asserts. Feel free to add your own ;)*/
}

For easier to read code or contributions, check out the google code project here.

 

Android App Suggestions

As new Android users, many of my friends and family have asked me what Android applications I would suggest/endorse. Here is a brief list of some of my favorite and/or most used applications.

Android App Suggestions

 - Vinny DeFrancesco

Audio/Alerts/Volume

Browser Replacement

Keyboard Replacement:

Password Management:

Bookmark Managment:

Music Players/Streamers/Management

Photos

Notes

MISCELLANEOUS

Remote Desktop

File Managment

 

Audio/Alerts/Volume

AudioManager Pro

Market Link: http://goo.gl/etWZ5

If there is one Android application I can’t live without, this is it. This app puts all of your volume management sliders in one place. I don’t know why Android does not do this by default. I was seriously surprised that I needed to adjust the volume in several different screens for Media, Calls, Alerts, and alarms. This app also allows you to create preset ‘shortcuts’ and put those on your home screen for one click access to a particular volume profile. I can’t recommend this app highly enough and it’s Google’s fault for not having a better way to manage Android’s volume.

Browser Replacement

Dolphin Browser™ HD

Market Link: http://goo.gl/uZ9qI

Yes, there was a privacy fiasco with the dolphin browser. Yes, it has been fixed. The Lastpass and Xmarks integration make this my goto choice for the default Android browser replacement. (more on LastPass and Xmarks below)

Info about previous privacy concerns: http://goo.gl/fWCmX

Keyboard Replacement:

SwiftKey X Keyboard

Market Link: http://goo.gl/eTLtl

If you don’t like the stock keyboard, give Swiftkey X a try.. After using it for a few weeks it’s word suggestion begins to get scary accurate. (and sometimes hilarious)

Password Management:

LastPass Password Mgr Premium

https://lastpass.com/

Market link: http://goo.gl/vsVfL

If you’re saving your password in your browser, you’re doing it wrong. If you’re writing your passwords down on pieces of paper, you’re doing it wrong. I’ve wrote about this before. If you want more info I’ll be happy to again. There is no good reason NOT to use lastpass as your defacto password management tool. (Unless you’re extremely distrustful of cloud services, in which case you get a pass; pun intended, suck it. That being said, you should be utilizing some sort of password management like KeePass; http://keepass.info). Lastpass securely stores your passwords for you and makes them accessible wherever you have an Internet connection. (And even when you don’t, via your mobile phone). You need to set up a lastpass account and purchase a “premium” account. (Last time I checked, it was $12 a year). Purchasing premium accounts allows the Lastpass company to run the secure servers that syncs up your password data. In my opinion, $1 a month is well worth the service.

Bookmark Managment:

Xmarks for Premium Customers

http://www.xmarks.com

Market Link: http://goo.gl/PZg6y

Xmarks syncs your bookmarks across any browser with xmarks installed. This includes the Dolphin HD Browser. You’ll need to set up an Xmarks account via their website. Xmarks was recently acquired by lastpass. I believe your lastpass premium subscription also covers the Xmarks premium subscription. I need to double check this.

Music Players/Streamers/Management

Disclaimer: I prefer “cloud storage” music players like Google Music and Amazon MP3. I like to support the Artists I listen to by digitally purchasing their work. If you’re into online radio streaming, check out Pandora, Slacker and/or Spotify. I do however, have not so nice things to say about Pandora, so I’ll mention them here for no other reason than to rant.

Pandora

http://www.pandora.com

Market Link: http://goo.gl/awLQD

If you’re into customizable streaming internet radio, Pandora is the way to go … that is if you can deal with their limited number of song skips (Even with a paid subscription). Pandora never fails to disappoint at the most inopportune moments; Like on that last mile of a really long run, Pandora will inevitably decide to play something like ColdPlay’s “Yellow”,  completely ruining  any gained momentum you might have hoped to achieve for the last bit of your workout.  “No worry” you think, I’ll just skip it … WRONG. You’ve run out of skips. You are now relegated to the most de-motivational music on the planet for the worst run of your life. F^&* you pandora.

Google Music

https://music.google.com/music

Market Link: http://goo.gl/4gvLl

Google music stores all of your music “in the could” (on Google’s servers) and allows you to stream that music to your phone via WiFi or over your cellphone network. If, like me, you have too much music to cram onto your phone, Google Music is a good solution. So is Amazon’s MP3 player/Cloud Player/Storage. More on Amazon below.

Amazon Mp3

https://www.amazon.com/gp/dmusic/mp3/player

Market Link: http://goo.gl/wKvUO

Before Google Music was widely available to the public, Amazon MP3 offered what is now essentially the same service. I find myself still using the Amazon MP3 app more often than Google Music, primarily because it’s so easy to purchase music right from my phone. Additionally, when you make a music purchase from your phone (or computer), the purchase gets synced to your Amazon Cloud Player. What this means is that you do not need to download your music to your phone (Though you can if you wish) but instead can stream it from Amazon’s servers. for $20 a year you get unlimited music storage and 20GB of ‘cloud storage’ More info: http://www.amazon.com/b?ie=UTF8&node=2658409011

Photos

The default Gallery app on Android sucks. Some manufacturers have rolled their own photo management applications into Android. If that works for you, great. You can skip this. If you find that you need tighter Picasa/Flickr integration or want to try something else, then I suggest;

JustPictures!

Market Link: http://goo.gl/YuJ0Q

https://picasaweb.google.com/home

I’ve found this app to be faster and easier to use than any of the default photo applications installed on Android phones. I used Picasa (and now Google+) almost exclusively for my online photo storage and sharing. I did have a little fling with Flickr when I first purchased my Droid Incredible (because that’s the only service HTC’s Sense UI would allow you to interact with), but since rooting my device, I’m thankfully back to Picasa (online photo storage and management. Interfaces directly with Google+) .

Notes

AK Notepad

Market Link: http://goo.gl/FVU5s

Android does not have a standard ‘notes’ application like Apple iOS. I don’t know why. AK Notepad is essentially the same as Apple’s Notes. It’s simple and it works. Great for shopping lists, Todo lists and murder lists. … oh, are you still reading this? 

MISCELLANEOUS

Kitchen Timer

Market Link: http://goo.gl/BoV9v

A simple countdown/up timer. Plenty of options for what it is. Again, your phone may already have a timer app via HTC Sense or Moto-crap … i mean Moto-blur. If you’re running stock Android, this app gets the job done.

Relax and Sleep

Market Link: http://goo.gl/y7qoJ

I require this app to function normally. Seriously. I can’t sleep without some sort of white/ambient noise equivalent. This app is real simple and does exactly what I need for generating background sleep noise. Additionally, it has a timer on it as well. You can configure it to play a noise or fade itself out when the timer ends. This is really useful for meditation sessions (if you’re into that kind of thing).

Kindle

Market Link: http://goo.gl/KrlUq

I don’t mind reading books on a digital display. Then again, I sit in front of a computer screen almost 24 hours a day. let’s just say I’ve ‘adjusted’. Even if you don’t have a kindle, the kindle app and Amazon’s ebook services are very cool. I’ve been known to own both a hard copy and a digital copy of some literature. I can leave the hardcopy on my shelf for collection purposes and “Look at what I read!” purposes, while making notes, highlights and links in my digital versions. There’s also just something cool about being able to carry an entire library in your back pocket.

Pure Grid calendar widget

Market Link: http://goo.gl/XoqCU

Android’s default Calender widgets are not good. They aren’t very configurable and they look like crap. This application solves both of those problems. At first, it can be a little confusing to set up because it is so highly configurable. Once you have it the way you like it, you’ll wonder how you lived without it.

Netflix

Market Link: http://goo.gl/8yFb6

If you don’t know what Netflix is then you probably don’t have any business reading this article. I suggest you read this: http://en.wikipedia.org/wiki/Internet and this

http://www.youtube.com/watch?v=cZC67wXUTs

Just remember, this is NOT a big truck.

Remote Desktop

PhoneMyPC

Market Link: http://goo.gl/TqgGc

I’ve been using this application since it was in Beta. Most of the time, it works. Most of the time. even over 3g connections. I’ve never had 100% success rates with any remote desktop software except for GoToMyPc, and unfortunately, that is not available on Android and is also a subscription service.

Splashtop Remote Desktop

Market Link: http://goo.gl/AaoZo

I need to admit that this application works A LOT better on my iPad than on my Android phone. However, if for some reason I can’t connect to my computer using GotoMyPc or PhoneMyPC, I’ll try this guy.

File Managment

ASTRO File Manager

Market Link: http://goo.gl/gHTWg

If you ever need to view the contents of your Android’s filesystem this application works very well. There is a free version available. If you need more features like Network share access and root prvileges, take a look at:

ES File Explorer

Market Link: http://goo.gl/TqgGc

A file explorer with advanced features like FTP, Root and network shares.

 

 

Moving your Google Desktop Database Location

I’ve recently realized that my windows ‘user profile’ was being backed up across my work network in order to support a ‘roaming profile‘. The roaming profile allows me to log on to any machine within my company’s network and have the same desktop, links, browser favorites, and mapped network drives (as well as several other application specific settings). My problem was that this user profile was being synced across the network any time I logged in or off of my local machine and the process was taking FOREVER; Up to an hour and 15 minutes at times. This was attributed to the large (3.5GB) google desktop database folder as well as some other Java development environment cache indices. All in all, I was transferring over 8GB across the network every time I logged in or off. No wonder it was taking forever for my machine to startup or shutdown!

The obvious solution was to move all of the large collections of Google Desktop data out of my user profile. But wait, There is no setting within the Google Desktop preferences menu! To the windows registry!

Before we go mucking with registry settings, make sure to shutdown google desktop. We’ll also want to move the google desktop database location out of our user profile. By default, the Google Desktop database folder resides in %LOCALAPPDATA%\Google\Google Desktop\.  You can find more information about backing up the Google Desktop Database here. http://desktop.google.com/support/bin/answer.py?answer=13799. For now, we’re just interested in getting it out of our profiles directory and putting it in a location that does not get transferred across our network. Within the %LOCALAPPDATA%\Google\Google Desktop\ directory is another directory whose name is comprised of random numbers and letters. This is the directory that you want to move to another location. To keep things simple lets say we moved it to c:\google desktop\database\<as763gdas>

Now that we’ve moved the physical location of the database elsewhere, it’s time to make a simple registry change.

Before we modify the windows registry settings it’s always a good idea to back them up. Before we back them up, we need to get into a registry editor. press windowskey+r. This will open up the ‘run’ dialog box.

Type in ‘regedit’ and hit enter. now select the export option from the file dropdown menu:

Call the registry backup file anything you’d like and save it to a location you will remember.

Now, within the regedit window navigate to HKEY_CURRENT_USER\Software\Google\Google Desktop. The registry key that you want to change is called “data_dir “. Change the value of that key to the new location of the Google Desktop database folder.

Now, just restart Google Desktop and presto, your Google Desktop Database woes are long gone.  :)

 

Scraping HTML using Java Servlets and TagSoup

I’ve been working on a simple java project to scrape some data from vendor websites in order to compare prices. I found a neat little library called “TagSoup” to help parse through the HTML tags returned from my URL connection. I ran into a few hiccups along the way which I figured might be worth documenting not only for my own sanity but hopefully for other code monkeys searching the net for solutions to these problems.

First of all, good luck finding any clearly written usage guides for the TagSoup library.  I was able to find a nice writeup over at HackDiary written in 2003 that gave me a nice starting point. The code provided there looks like this:

1
2
3
4
5
6
7
8
URL url = new URL("http://example.com");
// build a JDOM tree from a SAX stream provided by tagsoup
SAXBuilder builder = new SAXBuilder("org.ccil.cowan.tagsoup.Parser");
Document doc = builder.build(url);
JDOMXPath titlePath = new JDOMXPath("/h:html/h:head/h:title");
titlePath.addNamespace("h","http://www.w3.org/1999/xhtml");
String title = ((Element)titlePath.selectSingleNode(doc)).getText();
System.out.println("Title is "+title);

This Implementation worked just fine for me, though I did run into a couple of issues.

  1. I am building my projects using Apache-maven so I needed to add a dependencey for a newer version of Saxon than what is shipped with the JDK6 library.
    1. 1
      2
      3
      4
      5
      
      <dependency>
          <groupId>net.sf.saxon</groupId>
          <artifactId>saxon</artifactId>
          <version>8.7</version>
      </dependency>
  2. The default User-Agent that was being added to my GET headers was ‘Java/1.6.0_13′. This was causing some of the pages I needed to scrape to return back an 403 Forbidden error.Changing that was easy enough if I was not running this code from within a servlet.
    1. 1
      2
      3
      
      System.setProperty("http.agent", "Mozilla/5.0 "
              + "(Windows NT 6.1; U; ru; rv:5.0.1.6) "
              + "Gecko/20110501 Firefox/5.0.1 Firefox/5.0.1");

    However, When running this code from within a servlet using GlassFish3, I got stuck with the same 403 forbidden errors. I was able to resolve that with the following code changes:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    
    URL url1 = new URL("http://example.php");
    HttpURLConnection conn = (HttpURLConnection) url1.openConnection();
    conn.addRequestProperty("User-Agent", "Mozilla/5.0 "
            + "(Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) "
            + "Gecko/20061204 Firefox/2.0.0.1");
    // build a JDOM tree from a SAX stream provided by tagsoup
    SAXBuilder builder = new SAXBuilder("org.ccil.cowan.tagsoup.Parser"); 
    BufferedReader reader = new BufferedReader(
            new InputStreamReader(conn.getInputStream()));
    Document doc;
    String price = null;
    doc = builder.build(reader);

    Notice that I’m now passing the build() method a BufferedReader object instead of the Document object.

    For whatever reason, setting the system property from the servlet was not getting the job done.

  3. TagSoup seems to expect an ‘h:’ name space in front of all of the XPath elements. I was using the Firebug plugin for Firefox. The String returned from the ‘Copy XPath’ feature needed to be modified to include ‘h:’ after every node. Additionally, the ‘Tidying up’ that TagSoup performs seemed to have removed any ‘/tbody’ nodes. Those needed to be removed from my XPath string. here is an example of the XPath string I needed to use in order to grab the Silver Bid/Ask price from ‘http://bullion.nwtmint.com/silver_panam.php’:
    1
    2
    3
    
    JDOMXPath titlePath = new JDOMXPath("/h:html/h:body/h:table/h:tr[3]"
            + "/h:td/h:table/h:tr/h:td[3]/h:table/h:tr[3]/h:td/h:table"
            + "/h:tr[2]/h:td[2]");

I hope this information is able to help someone else out there.
 

Restart or shutdown Windows 7 from Remote Desktop

By default, when you are remote desktop-ed into a machine, the start menu does not show an option to shutdown or restart. These options are not even shown in the Task manager. A quick google search turned up a reply from Mark L. Ferguson, Moderator on Social.Answers.Microsoft.com.  

here’s the skinny:

press the windows key + r

Type in shutdown -r -t 0 <- That’s a Zero

Hit Enter

the ‘-r’ parameter tells shutdown.exe to ‘restart’ and the ‘-t 0′  tells the application to show zero pop up warnings

Windows 7 Won’t let Me Delete My Files or Folders!

In trying to cleanup my computer after installing Windows 7, I had begun running into a reoccurring problem where windows would not let me delete certain folders. I was getting a “_blank_ is open in another program” messages. I had tried programs like FileAssassin and LockHunter to no avail.

The solution that worked for me was to remove some of the hidden system files in the folders I was attempting to delete. I would have thought that both fileAssassin and Lockhunter would have been smart enough to do this sort of thing automagically… (yes, that’s a word, but only on the internets) but alas, I had to find the workaround myself.

From your windows explorer windows

  • goto Tools -> Folder Options…
  • Select the “View” tab
  • in the Advanced Settings box, select “Show hidden files, folders and drives
  • folder-options

After you can see the hidden files and folders, look for files such as

  • desktop.ini
  • Thumbs.db
  • AlbumArt_{A588129E-BA58-416A-85E5-B0A8E2BE38F9}_Small.JPG

If you truly want to delete these folders, remove these files. I did not have a problem simply highlighting them and pressing delete, however, ironically, you may need to use one of the above mentioned programs to unlock the files before you can delete them. After these files are removed you should not have trouble deleting the parent folders.

Hope this helped.