A key part of the missile launcher control is the ability to "see" thru the webcam. in this project, motion detection is defined as the delta between a reference image, and the image captured by the webcam at any given time.

Once again, browsing around the net looking for code samples on how to use webcams in C# can give you A LOT of options. And once again, I'm not claiming the the code I wrote is better than any other sample out there, but unfortunately I needed something simple that could enable me to do two things:

• capture snapshots: I need to capture reference images and compare them against the streaming video from the cam
• stream video: the app should contain a window that receives the cam stream

So I wrote a simple wrapper class - WebcamWrapper - that exposes three methods:

• OpenCam - attaches a window handles to the web cam device
• CloseCam - release all handles
• CaptureWebImage - returns a bmp representing a snapshot of the cam (I just realized this is a really bad name...)

The code has tons of references to system DLLs, but the most important is certainly avicap32.dll - capCreateCaptureWindowA is the function that connects the webcam to a window in the app. The window I use is actually one of the Image controls in the form, so the final result is the video stream showing where it should... The Image control is passed to the class constructor, which then saves it as a private object.

static PictureBox picCapture = null;

/// <summary>
/// constructore
/// </summary>
/// <param name="formPicture">form object that receives the stream</param>
public WebcamWrapper(PictureBox formPicture)
{
    picCapture = formPicture;
}

/// <summary>
/// Close the stream and disconnect the device
/// </summary>
public void CloseWebCam()
{
    SendMessage(hHwnd, WM_CAP_DRIVER_DISCONNECT, iDevice, 0);
    DestroyWindow(hHwnd);
}

/// <summary>
/// Open the device stream
/// </summary>
/// <returns>true if success, false otherwise</returns>
public bool OpenWebCam()
{
    int iHeight = picCapture.Height;
    int iWidth = picCapture.Width;

    hHwnd = capCreateCaptureWindowA(iDevice.ToString(), WS_VISIBLE + WS_CHILD, 0, 0, picCapture.Width, picCapture.Height, picCapture.Handle.ToInt32(), 0);
    int camAvailable = SendMessage(hHwnd, WM_CAP_DRIVER_CONNECT, iDevice, 0);

    if (camAvailable > 0)
    {
        SendMessage(hHwnd, WM_CAP_SET_SCALE, -1, 0);
        SendMessage(hHwnd, WM_CAP_SET_PREVIEWRATE, 66, 0);
        SendMessage(hHwnd, WM_CAP_SET_PREVIEW, -1, 0);
        SetWindowPos(hHwnd, HWND_BOTTOM, 0, 0, picCapture.Width, picCapture.Height, SWP_NOMOVE + SWP_NOZORDER);
        return true;
    }
    else
    {
        DestroyWindow(hHwnd);
    }
    return false;
}

/// <summary>
/// Captures the desktop work area to a bitmap image
/// </summary>
/// <returns>bitmap image of desktop</returns>
public Bitmap CaptureWebImage()
{
    int x;
    int y;
    int width = picCapture.Width;                                                                                            // width of desktop
    int height = picCapture.Height;                                                                                           // height of desktop
    IntPtr picHWND = picCapture.Handle;     //

    x = 0;
    y = 0;

    IntPtr wndHDC = GetDC(picHWND);                             // get context for window

    // create compatibile capture context and bitmap
    IntPtr capHDC = CreateCompatibleDC(wndHDC);
    IntPtr capBMP = CreateCompatibleBitmap(wndHDC, width, height);

    //  make sure bitmap is valid
    if (capBMP == IntPtr.Zero)
    {
        ReleaseDC(picHWND, wndHDC);                   //   release window context
        DeleteDC(capHDC);                             //   delete capture context
        return null;                                  //   return null bitmap
    }
    else
    {
        // select compatible bitmap in compatible context
        // cpy window context to compatible context
        // select previous bitmap back into compatible context
        IntPtr prvHDC = (IntPtr)SelectObject(capHDC, capBMP);

        BitBlt(capHDC, 0, 0, width, height, wndHDC, x, y, 13369376);
        SelectObject(capHDC, prvHDC);

        // create GDI+ bitmap for window
        Bitmap bmp = System.Drawing.Image.FromHbitmap(capBMP);

        // release window and capture resources
        ReleaseDC(picHWND, wndHDC); // release window context
        DeleteDC(capHDC);               // delete capture context
        DeleteObject(capBMP);           // delete capture bitmap

        // return bitmap image to user
        return bmp;                                                                                                                                       // return bitmap
    }
}

The rest is just bitmap manipulation...

Here's the pointer to the code:

Next blog will cover the algorithm I used to detect motion, and to define what actions to take to take aim at the intruder...

Enjoy!

Mais uma vez, buscar exemplos de código para web cam em C# pode dar-lhe muitas opções. E mais uma vez, eu não me estou reivindicando o o código que eu escrevi seja melhor do que nenhuma dessas outras opções, mas infelizmente eu preciso de algo simples que me permita de fazer duas coisas:

• snapshots: Eu preciso de capturar imagens da referência para compará-las com o video stream 
• stream video: a aplicaçao deve conter uma janela que conectada ao video da web cam

Assim eu escrevi a uma classe simples - WebcamWrapper - que expoem três métodos:

• OpenCam - conecta um pointeiro para uma janela ao device da camera 
• CloseCam - libera os recursos alocados
• CaptureWebImage - retornos um bmp que representa um snapshot da cam

O código tem toneladas de referências a DLLs de sistema , mas a mais importante é certamente avicap32.dll - capCreateCaptureWindowA é a função que conecta a camera web a uma janela na aplicaçao. A janela que eu me uso é  na realidade um dos controles da imagem no formulário, assim que o resultado final é a exibição do video stream no lugar devido… O controle da imagem é passado ao construtor da classe, que salva uma referencia em um objeto private.

O resto do codigo é manipulacao de bitmap... Nada de complicado...

(Nota do autor: parte do texto acima foi traduzido por um PlugIn para o Live Writer ;-))

So Google released a translation bot that works with their Messenger like app yesterday, and I was surprised to see how similar it is to my little project :-). It's cool to see that the bots are getting momentum and finally getting rid of the spam reputation, but frankly it's even cooler to know that I beat them by 6 months or more, working  with a team of 1 (= me, btw);-). Bragging right much???

Pois e'... Nao e' que ontem a Google lancou um bot de traducao? Pelas telas que andei vendo parece com a versao 0.1 do meu projetinho, acreditem ou nao... E' legal ver os bots ganhando fama, e talvez se livrando do estigma de spam. E convenhamos e' muito mais legal ainda saber que eu lancei o meu projeto pelo menos seis meses antes, e com toda a minha equipe do "eu sozinho" trabalhando so' algumas horas de madrugada.

Parece besteira, mas que e' bom e'...

Yet another post from the series "this has nothing to do with Machine Translation" :-). But c'on! It IS an interesting topic... Well, at least it's interesting for me...

So last month I bought one of those USB missile launchers at Fry's for something like 20 bucks or so... I wanted to attach a web cam to it and make it sort of detect people and shoot automatically - I know, I'm not 9 anymore but it's in fact a pretty cool project! Think about it: you have device control, webcam capture, pattern recognition, etc, etc etc... Several excuses to try to explain why I'm spending time playing with a silly toy.

Anyway, once I got the little thing installed and running with the lame app that comes with it, I started looking around for references and code samples. The first thing I found out is that most (if not all) USB toys are installed as HID (Human Interface Device) artifacts. That means that it supports a rather simple protocol, which is actually good news.

I found a cool article here: http://www.codeproject.com/KB/cs/USB_HID.aspx?df=100&forumid=398968&exp=0&select=2200544&tid=2200544. The library was quite straight forward and easy to use. After snooping around a bit I found out the codes for the device I bought - the Talon Model: TT-782 maps to Vendor ID = 0A81 and Product ID = 0701. This is quite important because the device drivers are treated as files by the OS (note: this is an over simplified explanation). In my case, the full path that maps to the launcher is:

\\?\hid#vid_0a81&pid_0701#7&36f50288&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}

Don't panic... You don't actually need to know all of this - the library is to expose an object - UsbHidPort - that abstracts most of the complexity from you - just provide the right Vendor ID and Product ID. The library even includes a friendly custom control that you drag and drop to your form. In our case, the control initialization code will look like this:

//
// usb
//
this.usb.ProductId = 2689;
this.usb.VendorId = 1793;
this.usb.OnSpecifiedDeviceRemoved += new System.EventHandler(this.usb_OnSpecifiedDeviceRemoved);
this.usb.OnDeviceArrived += new System.EventHandler(this.usb_OnDeviceArrived);
this.usb.OnDeviceRemoved += new System.EventHandler(this.usb_OnDeviceRemoved);
this.usb.OnDataRecieved += new UsbLibrary.DataRecievedEventHandler(this.usb_OnDataRecieved);
this.usb.OnDataSend += new System.EventHandler(this.usb_OnDataSend);
this.usb.OnSpecifiedDeviceArrived += new System.EventHandler(this.usb_OnSpecifiedDeviceArrived);

The IDs were converted to decimal. To use the device you would need to open it, like this:

private void btn_ok_Click(object sender, EventArgs e)
{
    try
    {
        this.usb.CheckDevicePresent();
    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.ToString());
    }
}

If the library successfully connects to the device, it will raise OnSpecifiedDeviceArrived, which you can trap to do whatever you need:

private void usb_OnSpecifiedDeviceArrived(object sender, EventArgs e)
{
    this.lb_message.Items.Add("My device was found");

    //setting string form for sending data
    string text = "";
    for (int i = 0; i < this.usb.SpecifiedDevice.OutputReportLength - 1; i++)
    {
        text += "000 ";
    }
    this.tb_send.Text = text;
}

Alright, so now the device is connected and ready to be used... Then what? Well, now you need the commands to send. I experimented a little and found the following codes:

// launcher control codes
const string CMD_DOWN = "00 01";
const string CMD_UP = "00 02";
const string CMD_LEFT = "00 04";
const string CMD_RIGHT = "00 08";
const string CMD_FIRE = "00 16";
const string CMD_STOP = "00 32";

So this device responds to two word command sequences, that always start with 0. To send the sequence to the launcher, use something like this:

// send the command string to the HID device
//
void SendCommands(string cmdString)
{
    try
    {
        string[] arrText = cmdString.Split(' ');
        byte[] data = new byte[arrText.Length];
        for (int i = 0; i < arrText.Length; i++)
        {
            if (arrText[i] != "")
            {
                int value = Int32.Parse(arrText[i], System.Globalization.NumberStyles.Number);
                data[i] = (byte)Convert.ToByte(value);
            }
        }

        if (this.usb.SpecifiedDevice != null)
        {
            this.usb.SpecifiedDevice.SendData(data);
        }
        else
        {
            MessageBox.Show("Sorry but your device is not present. Plug it in!! ");
        }

    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.ToString());
    }
}

Once the commands are sent to the device, the usb object will raise OnDataSend, which in our case doesn't do anything...

That's it! That's all you need to control the basic movements of your Talon TT-782, thanks to the fine folks that wrote this nice library... All credits go to them... Also, check this article - http://www.vsj.co.uk/articles/display.asp?id=600 - to know more about HIDs and USB devices in general.

Next blog will cover the webcam interface. Stay tuned!

Se vc e' como eu e gosta de projetos legais mas que nao servem pra nada, esse deve chamar sua atencao... Eu comprei um desses brinquedinhinhos que se conectam na porta USB do micro - um lancador de misseis - e grudei uma webcam da Logitec no treco. A ideia era desenvolver um "sistema de seguranca" que detectasse qq mudanca no ambiente monitorado, e "atacasse" o intruso com os poderosos misses de isopor.

Pra comecar o projeto eu tive que identificar as caracteristicas do dispositivo. Descobri que a maioria dos brinquedinhos USB sao do tipo HID (Human Device Interface). Procurei um pouco e achei uma biblioteca que tem ate' um Custom control para o Visual Studio e que funciona muito bem - chequem esse artigo aqui http://www.codeproject.com/KB/cs/USB_HID.aspx?df=100&forumid=398968&exp=0&select=2200544&tid=2200544.

De posse da library, ficou mais facil... Com um pouco de esforco deu pra identificar os codigos do Vendor e do Produto que sao necessarios para utilizar o dispositivo. Nesse caso o Talon TT-782 e' instalado com Vendor ID = 0A81 e Product ID = 0701 (cheque o codigo na versao em ingles).

Para utilizar o dispositivo, vc tem que abri-lo tal como se fosse um arquivo. No nosso caso, a library ja' tem todas as funcoes necessarias para manipular o device, portanto basta passar os codigos corretos e chamar CheckDevicePresent(). Ao conectar com o lancador de misseis, o objeto chama OnSpecifiedDeviceArrived indicando portanto que tudo funcionou como devia.

Nesse ponto temos o lancador conectado ao codigo e pronto para fazer alguma coisa... Mas e dai? Como e' que o treco mexe? Bom, vc vai precisar mandar codigos de dois bytes que representam comandos do lancador. Eu testei varias combinacoes, e descobri o seguinte:

DOWN = "00 01"
UP = "00 02"
LEFT = "00 04"
RIGHT = "00 08"
FIRE = "00 16"
STOP = "00 32"

Para enviar os comandos, crie uma funcao como SendCommands (ver acima), que basicamente quebra a sequencia de codigos em bytes e envia o array para o device.

E e' isso... Parece complicado, mas gracas ao pessoal que desenvolveu a biblioteca, fica muito facil...Vale a pela ler o artigo deles com cuidado, e tambem esse aqui: http://www.vsj.co.uk/articles/display.asp?id=600 . Se vc preferir a versao traduzida, clique aqui: http://www.windowslivetranslator.com/BV.aspx?lp=en_ja&a=http%3a%2f%2fwww%2evsj%2eco%2euk%2farticles%2fdisplay%2easp%3fid%3d600 

No proximo blog vou descrver a interface com a web cam. Fiquem ligados!

A couple of weeks ago I was toying with the idea of putting together some app that would translate screen shots. Not a very practical app, but I'm imagining that someone may have problems using a non-localizable application, and eventually knowing what the menu items mean could be might be helpful. I could have done it in many different ways, I know... But given that I wanted to cover dynamic web pages - like Flash apps - and other non-windows based content, capturing the screen and using an OCR seemed to be an adequate option.

That said, I was looking around and found a plethora of examples of OCR controls you can buy out on the web, but to my surprise I also found references to an OCR feature that ships with Office 2007! That's right: if you have Office 2007 Professional you can use its own OCR control. Many code samples I found worked great, but some didn't so I ended up building my own, and integrating it with a test build of our translator.

The control is really simple to use. All you have to do is add a COM reference to "Microsoft Office Document Imaging 12.0 Type Library" like below:

Then all you have to do is make sure your file uses the MODI Interop... The recognition requires either a BMP or TIFF image, so you have to make sure the file you load is in the correct format. In the demo video you'll see that I added a step to convert a loaded image to BMP, and only then I submit it for recognition. The OCR related portion of the demo is as follows:

namespace Sample
{
    using System;
    using System.Windows.Forms;
    using System.Threading;
    using SpeechLib;
    using System.Drawing;
    using MtService;
    using System.Collections.Generic;
    using System.Globalization;
    using MODI;

...

private void btnOCR_Click(object sender, EventArgs e)
{
    myOCR = new MODI.DocumentClass();
    myOCR.Create(picOCR.Tag.ToString());

    // add a handler to trap the tracking progress (it doesn't go all the way to 100% sometimes)
    myOCR.OnOCRProgress += new MODI._IDocumentEvents_OnOCRProgressEventHandler(this.ShowProgress);

    MiLANGUAGES langSelection;

    // this example currently only works with English
    langSelection = MiLANGUAGES.miLANG_ENGLISH;

    // this is a synchronous call (events are fired in the background)
    myOCR.OCR(langSelection, true, true);

    string myText = String.Empty;
    myLayout = ((MODI.Image)myOCR.Images[0]).Layout;

    string[] lines = myLayout.Text.Split('\n');
    txtSourceText.Text = String.Empty;

    for (int index = 0; index < lines.Length; index++)
    {
        txtSourceText.Text += lines[index] + Environment.NewLine;
    }

    btnSay.Enabled = true;
    btnFilter.Enabled = true;
    btnEraser.Enabled = true;
}

// here's the callback function from the OCR itself
public void ShowProgress(int progress, ref bool cancel)
{
    lblStatus.Text = progress.ToString() + "% processed.";

}

So it's really simple, actually. The rest of the code just deals with loading pictures from disk or pasting them from the clipboard, and there's also the interface to the translation services. The demo video shows that the code works for both a "good" image loaded from disk, and for a screenshot pasted from the clipboard.

I was pleasantly surprised with the overall quality of the screenshot recognition. The OCR gets most of the content right. Unfortunately, I can't say the same about low quality pictures with text - no luck there. However, it also works pretty well with cartoons :-).

Enjoy the sample, and the demo...

NOTE: I had to force 32 bits builds in my Vista 64 bits desktop... the OCR version I have only supports 32 bit apps

Eu andei avaliando umas opcoes para implementar um tradutor de telas, e procurando de um lado e de outro acabei por dar de cara com um componente que faz parte do Office Professional 2007 que eu nao tinha a menor ideia que existia:  Microsoft Office Document Imaging 12.0 Type Library. E' basicamente um OCR que ja' vem instalado no Office (serio, nao custa nada a mais nao...). Muitos exemplos que eu achei funcionavam, mas outros nao rodavam de jeito nenhum. Resolvi entao simplificar as coisas ao maximo, e escrever minha propria versao...

O codigo esta' ai' no blog em Ingles - e' so' copiar e alterar as variaveis. Nao tem truque: adicione uma referencia COM apontando para "Microsoft Office Document Imaging 12.0 Type Library", adicione "using MODI;" no inicio do programa e faca bom proveito! O objeto requer que a imagem passada durante sua criacao (.Create) seja BMP ou TIFF. Na demo vcs podem perceber que eu adicionei um step que converte e grava a imagem carregada em BMP. A ativacao do reconhecedor e' sincrona, ou seja e' so' passar os parametros, chamar o metodo (.OCR) e pronto. O resultado vem na forma de um array de paginas com frases. Eu adicionei um event handler para captura os eventos de OnOCRProgress, mas e' so' pra' fornecer algum tipo de feedback visual pro usuario (nao altera em nada o resultado do reconhecimento). O resto do codigo da demo e' so' firula: interface com o tradutor e uma opcao para "ler em voz alta" o texto reconhecido.

O filminho da demo mostra que o codigo funciona tanto para imagens de boa qualidade lidas de disco quanto para imagens capturas via ALT+PrintScreen no Clipboard. Mas nao tenham esperancas com imagens da web - baixa qualidade nao gera nada legivel.

Mandem mensagens se tiverem problemas... Nao deve ser dificil botar pra funcionar fazendo as devidas alteracoes, mas nunca se sabe...

ATENCAO: eu tive que forcar a compilacao em 32 bits. O controle que esta' instalado na minha maquina nao roda em 64 bits.

Maybe it's the Holiday season or maybe it's just the dynamic of the services shipping cycle, but no matter after the crazy sprint to ship on time things are finally cooling down (actually maybe a little too much, if you ask me...). This month's release is going to be basically centered around some backend issues and on some performance enhancements - nothing exciting from a client side perspective.

That said, I'll be blogging about other stuff I play with - as is other code artifacts that someone else may find interesting... I found out a long ago that mining the web for code samples and explanation may be tough, so I guess it's my turn to share what I can. I'll isolate these entries with some tags, like 'Random Topics' or something alike to make it easier for people to distinguish them from Machine Translation related stuff.

And finally: I got a little digital cam to record some demos. I'm still trying to get used to it so be kind, alright? I've attached a my first Soapbox movie here: a brief take of the "oh so famous" Microsoft Research Building 99 here in Redmond. Looks like a fancy shopping center, but it's actually an office building. And yes, there's a Starbuck's on the first floor :-)...

(Nota do editor: Esse negocio de escrever o blog em dois idiomas ta' comecando a ficar complicado... Bom, mas vamos la' :-))

Finalmende depois de muita ralacao para conseguir entregar o Live Translator sem estourar o prazo, as coisas estao se acalmando por aqui... A versao desse mes vai conter basicamente melhorias de performance e algumas correcoes de problemas no servidor, basicamente nada de interessante para a aplicacao.

Com isso, sobrou tempo para bloggar sobre outras coisas, nao necessariamente relacionadas com traducao automatica. Vou incluir exemplos (C#) sempre que puder - sou fan de compartilhar codigo - quem sabe pode ser util pra alguem por ai? Os titulos das entradas do blog vao ter um tag tipo "Random Topics" pra isolar das outras relacionadas com MT.

E finalmente: seguindo uma dica antiga do Galileu (http://on10.net/blogs/Galileu/) eu tomei vergonha na cara e comprei uma filmadora digital pequenininha pra gravar demos quando achar necessario... Como primeiro video, fui ate' o lobby interno do predio e filmei um pouquinho... Esse e' o predio novo da Microsoft Research aqui em Redmond - o famoso predio 99. Muito bonito, nao tem duvida - parece um shopping center de luxo - mas nao e' tao funcional quanto devia nao... Os escritorios sao menore e (pasmem) nao tem tomada suficiente pro monte de maquinas que eu tenho depaixo da mesa. Mas nao to reclamando de barriga cheia nao - o bicho ficou bonito...

Segue o video...

- Hello: activates the bot to translate from English to any of the available languages (you will be prompted to select one)

- Hallo: activates the bot to translate from German to either English or French (you will be prompted to inform your option)

- Oi: activates the bot to translate from Portuguese to English (no other language is available)

- 你好: activates the bot to translate from Chinese Simplified to English (no other language is available)

- こんにちは: activates the bot to translate from Japanese to English (no other language is available) *** I translated the sentences myself using MT, so if there's any Japanese speaker out there willing to help translate the dialogs in a more polite fashion, please send me a message!!!