A threaded Go language application with semaphores

After hearing about Go and its love of parallelism, I wanted to give it a try. I couldn’t think of any small projects that needed to be threaded until recently.  My employer uses an RPM system, but I’m using Ubuntu, which requires deb files.  Searching through the RPMs in a browser is a pain, so I wanted a small script that would search for me.  Here’s the script with server names changed.

package main

import (
  "bufio"
  "fmt"
  "http"
  "os"
  "regexp"
  "strings"
)

var servers = []string {
  "http://www.google.com/search?client=ubuntu&channel=fs&q=go+language&ie=utf-8&oe=utf-8",
  "http://golang.org/",
  "http://golang.org/doc/go_tutorial.html"}

var maxOpenRequests int = 3

var rpmRegexp = regexp.MustCompile("href=\"[^\"]+\"")

func main() {
  search := getSearchTerm()
  out := make(chan string)
  done := make(chan int)
  go printer(out)
  for _, url := range servers {
    go searchURL(search, url, out, done)
  }
  for i := 0; i < len(servers); i++ {
    <- done
  }
}

func getSearchTerm() string {
  args := os.Args
  if len(args) != 2 {
    die("Please enter one search term")
  }
  return args[1]
}

func die(message string) {
  fmt.Println(message)
  os.Exit(1)
}

func printer(out chan string) {
  for {
    fmt.Println(<- out)
  }
}

var requestSemaphore = make(chan int, maxOpenRequests) // Integer chanel with a maximum queue size

func searchURL(search string, url string, out chan string, done chan int) {
  requestSemaphore <- 1 // Block until put in the semaphore queue
  response, realURL, err := http.Get(url)
  if err == nil {
    bufferedReader := bufio.NewReader(response.Body)
    err = searchAll(search, bufferedReader, url, out)
    response.Body.Close()
  }
  if err != nil {
    die("Could not read from " + realURL + ":" + err.String())
  }
  <- requestSemaphore // Dequeue from the semaphore
  done <- 1 // Signal that function is done
}

func searchAll(search string, reader *bufio.Reader, httpRoot string, out chan string) (os.Error) {
  var error os.Error = nil;
  for {
    var line, err = reader.ReadString('\n')
    if err != nil {
      if err == os.EOF {
        break
      }
      return error
    }
    var hrefs = rpmRegexp.AllMatchesStringIter(line, 0)
    for href := range hrefs {
      start := strings.Index(href, "\"") + 1
      end := len(href) - 1
      packageFile := href[start : end]
      if strings.Index(packageFile, search) != -1 {
        out <- httpRoot + packageFile
      }
    }
  }
  return error
}

A quick overview

Go uses “channels” for communication among threads.

  1. The main thread creates two channels, one to receive search output and one to receive “done” responses when a child thread finishes.
  2. The main thread attaches the output channel to a simple function that prints to standard out.
  3. The main thread spawns one child thread per server, giving each the output and done channels.
  4. The main thread loops over the done channel once for each server, waiting until all requests are complete.
  5. Each child blocks immediately until space is available in a semaphore.
  6. The child performs the HTTP get, then searches the response for anything matching a regular expression.  Any matches are signaled back to the output channel.
  7. The child opens a space in the semaphore and signals to the main thread that it is done.

There isn’t much more to say.  The Go web site has great tutorials, so I don’t yet feel the need to write one.

By the way, if you’re searching for Go resources, be sure to search for “go language” and not “go”.  But then, that’s probably how you found this page 😉

package main

import (
“bufio”
“fmt”
“http”
“os”
“regexp”
“strings”
)

var servers = []string {
“http://apt.duncllc.com/dist/WS4.0/RedHat/RPMS/”,
“http://apt.duncllc.com/dist/WS4.0/RedHat/RPMS.extras/”,
“http://apt.duncllc.com/dist/WS4.0/RedHat/RPMS.java/”}

var maxOpenRequests int = 3

var rpmRegexp = regexp.MustCompile(“href=\”[^\”]+\””)

func main() {
search := getSearchTerm()
out := make(chan string)
done := make(chan int)
go printer(out)
for _, url := range servers {
go searchURL(search, url, out, done)
}
for i := 0; i < len(servers); i++ {
<- done
}
}

func getSearchTerm() string {
args := os.Args
if len(args) != 2 {
die(“Please enter one search term”)
}
return args[1]
}

func die(message string) {
fmt.Println(message)
os.Exit(1)
}

func printer(out chan string) {
for {
fmt.Println(<- out)
}
}

var requestSemaphore = make(chan int, maxOpenRequests) // Integer chanel with a maximum queue size

func searchURL(search string, url string, out chan string, done chan int) {
requestSemaphore <- 1 // Block until put in the semaphore queue
response, realURL, err := http.Get(url)
if err == nil {
bufferedReader := bufio.NewReader(response.Body)
err = searchAll(search, bufferedReader, url, out)
response.Body.Close()
}
if err != nil {
die(“Could not read from ” + realURL + “:” + err.String())
}
<- requestSemaphore // Dequeue from the semaphore
done <- 1 // Signal that function is done
}

func searchAll(search string, reader *bufio.Reader, httpRoot string, out chan string) (os.Error) {
var error os.Error = nil;
for {
var line, err = reader.ReadString(‘\n’)
if err != nil {
if err == os.EOF {
break
}
return error
}
var hrefs = rpmRegexp.AllMatchesStringIter(line, 0)
for href := range hrefs {
start := strings.Index(href, “\””) + 1
end := len(href) – 1
packageFile := href[start : end]
if strings.Index(packageFile, search) != -1 {
out <- httpRoot + packageFile
}
}
}
return error
}

Posted in Go at May 21st, 2010. No Comments.

Android application Green Mileage open sourced

Green Mileage is now an open source application at Google Code.  You can find the full source for most of the Android posts on this site in it at http://code.google.com/p/greenmileage/.  Just click the Source tab, then the Browse link.

The code needs a lot of cleanup, but is still probably worth a look if you’re new to Android.

Posted in Android at April 27th, 2009. 4 Comments.

Tutorial for creating an icon for an Android button: Part 2 of 2

This is part 2 of this tutorial.  Click here to see part 1.

  1. You should have two shapes in Inkscape.
    offsetshadowarrowarrowmask
  2. Now, you have to get these into the GIMP.  The easiest way is to take a screen shot.  To do that, in GIMP, go to File -> Acquire -> Screen Shot.  Take two shots.  Crop one to the shadowed image and the other to the black and white.
  3. Pick the shadowed image and add an opaque layer mask.  View the Layers dialog.  Right click the “Background” layer and click Add Layer Mask…
  4. Pick the black and white image.  Select the entire image and copy it.
  5. Pick the shadowed image and in the layers dialog, click on the mask, to the right of the image.
  6. Pick the shadowed image and paste.  You should see something like the image below.  Pick the move tool and drag the mask around until it exactly matches the shadowed arrow.
    alphaarrow
  7. Now, crop the image to the arrow shape.
  8. Right click on the floating layer and anchor it.
  9. Right click on the mask layer and apply it.
  10. Now, just save it!  Remember to save it as a PNG so that the empty parts are transparent.
    finalarrow1

I hope this helps anyone else who needs more than the standard Android icons.

More updates:

Will S. created a wireless icon.

Great tutorial, and very easy to follow, thanks.
I made a wifi icon, which can be found here:
http://digitalsquid.co.uk/files/2009/08/wifiicon.png
(you can use it if you want)

Posted in Android at January 25th, 2009. 12 Comments.

Tutorial for creating an icon for an Android button: Part 1 of 2

Update

The Android developer site has some great icon guidelines now.  I recommend taking a look.  If you have Photoshop or another program that supports vector shapes and drop-shadows, you can see the proper settings there.

Nick says

… you can do the whole thing in Inkscape without having to go into GIMP, by using a clip mask.

Duplicate your arrow, then select both this and the inverse arrow and do Object > Clip > Set. The inverse arrow will be clipped to only show the content contained within the arrow shape.

Introduction

For my Android application, Green Mileage, I created a number input dialog box.  One of the buttons in the dialog is a backspace key.  Right now, it’s just a button that says, “Back”.  When typing, it’s function is not immediately intuitive, so I wanted to replace it with an ImageButton.  To skip to the example, scroll down. Unfortunately, none of the Android icons look quite right.  I found an image of a red X and a trash can.  Neither would look like a backspace, so I decided to create my own.

Since Android is open source, I assumed that Google would have created some kind of template for button images.  I thought I would find a PSD, but after much searching, could not.  Rather than pay $1,000 for Photoshop, I decided to use open source applications to create the image.  Here’s how I did it.

Step 1: Get Inkscape and GIMP

Since this drawing will be made with vector graphics, it’s best to use a vector graphics program, so head on over to Inkscape’s web site and download it if you don’t already have it.  The post-processing will be done in GIMP, so grab a copy of that too.  Both have versions for Linux and Windows.

Step 2: Create the Inkscape drawing

The points on the arrow should be evenly spaced, so they shouldn’t be drawn by hand.  The arrow is really just a triangle with a rectangle dangling from its side.

  1. This tutorial is about drawing an arrow shape, but you can use most of these steps for any icon.
  2. First, let’s draw the arrow.  Pick the polygon tool: Polygon tool
  3. In the toolbar at the top of the screen, set these values.
    1. Corners: 3
    2. Spoke ratio: 0.5
    3. Rounded: 0
    4. Randomized: 0
  4. In the drawing canvas click and hold.  While holding the left mouse button, press and hold the control key.  That will help you draw the triangle so that the right side is vertical.
    triangle
  5. Select the rectangle tool and draw a rectangle for the right side of the arrow.  Make it overlap with the triangle.  Then, select both objects.  In the menu bar, click Object -> Align and Distribute.  Click the “Center on horizontal axis” button.
    trianglerectangle
  6. Now, with both objects selected, in the menu bar, click Path -> Union.
    arrow
  7. Android menu buttons have rounded corners and gradient fills.  Let’s apply those next.  With the arrow selected, in the menu bar, click Object -> Fill and Stroke.  Click the Stroke Style tab.  Select the round join and set the width to a high enough value that the corners look rounded.
  8. Select the Stroke Paint tab and pick the linear gradient.  The gradient will appear sideways, but we’ll fix it later.  Click Duplicate to create an Android button gradient.  Click Edit… Select the first stop at the top of the window, then at the bottom of the window, set its RGBA value to 666b66ff.  Set the second stop to b2afb2ff.
  9. Select the Fill tab, create a linear gradient and set it to the one you just created for the stroke.
  10. Close the Fill and Stroke dialog.
    horizontalgradientarrow
  11. To rotate the gradient, click the gradient tool, gradienttool.
  12. You’ll see two handles.  Drag the lighter side to the top of the arrow and the darker side to the bottom.  The handles will snap in to place.
    gradientarrow
  13. Android menu buttons also have an inner shadow.  Make a copy of the arrow away from the original arrow and draw a rectangle around it.  Make it a good amount larger.  Align the rectangle and the arrow the same way the triangle and rectangle were aligned before.  Then, select both the rectangle and the arrow and in the menu bar, select Path -> Exclusion.  Set the fill of the new shape to black and the path to the same gradient as the arrow, ensuring that the gradient minimum and maximum are aligned to the arrow, not the box.  Make a duplicate of the new shape and set it aside for use later.
    inversearrow1
  14. Select both the original and the new, inverse arrow and align them horizontally and vertically.
  15. Select the larger, inverse arrow and open its Fill and Stroke options,  move the blur slider to the right until it looks like a shadow.  Also adjust the alpha of the fill option until the arrow looks like it has an inner shadow.  Don’t worry if the space outside of the arrow doesn’t look right.  Then, since Android’s shadows are offset, move the inverse arrow down and to the right a bit.  The arrow keys are good for this.
    offsetshadowarrow
  16. Now, in the copy of the inverse arrow you made, set the stroke to white and the fill color to black.

Go to part 2

Posted in Android at January 24th, 2009. 12 Comments.