Category: C Sharp

Parsing HTML pages with Jsoup

Introduction

Recently I had an interest to retrieve contents from an HTML web page. Few suggestions on the project also led me to find on this area. Thinking even more, in order to keep the blog updated I thought of writing on the most interesting solution I found.

jsoup

Jsoup

Through the options I found, Jsoup has some powerful capabilities in extracting data from HTML pages. You can use regex kind of expressions to filter-out the elements in a HTML page. Through the following example you will see how to take the ‘body’ section of a web page and how to download the images in a HTML page.

How To Do It

If you are doing this in an IDE like Netbeans, you have to add Jsoup jar file to Libraries.

jsoup lib

Thereafter you can start coding. As I explained previously you will get the body text and images in a HTML page from the following code. I have explained the code using comments as far as I can. 🙂


import java.io.BufferedInputStream;
import java.io.ByteArrayOutputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

/**
 *
 * JSoup HTML parser
 *
 * @author BUDDHIMA
 */
public class JSOUP {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        try {

            /**
             * Web page URL you want to connect.
             * If lesser timeout, operation may fails
             */
            Document doc = Jsoup.connect("https://buddhimawijeweera.wordpress.com/2013/05/18/parsing-html-pages-with-jsoup/").timeout(300000).get();

            // Print out text contain in <body> section
            System.out.println(doc.body().text());

            // Take image urls
            Elements images = doc.select("img[src~=(?i)\\.(png|jpe?g|gif)]");

            // Image counter
            int i = 0;

            for (Element image : images) {
                try {
                    System.out.println("src: " + image.attr("src"));
                    String src = image.attr("src");

                    // Read images
                    URL url = new URL(src);
                    InputStream in = new BufferedInputStream(url.openStream());
                    ByteArrayOutputStream out = new ByteArrayOutputStream();
                    byte[] buf = new byte[1024];
                    int n = 0;
                    while (-1 != (n = in.read(buf))) {
                        out.write(buf, 0, n);
                    }
                    out.close();
                    in.close();
                    byte[] response = out.toByteArray();

                    // Save images
                    FileOutputStream fos = new FileOutputStream("borrowed_image-" + i + ".jpg");
                    fos.write(response);
                    fos.close();
                    i++;

                } catch (Exception e) {
                    System.out.println("Error in reading & storing images: "+e.getMessage());
                }
            }

        } catch (IOException ex) {
            System.out.println("Error: " + ex.getMessage());

        }
    }
}

Conclusion

Hopefully through the above example you can get a basic idea about Jsoup, but there’s a lot more you can do with Jsoup. References will definitely help you to go ahead 🙂

References

[1] Jsoup official page: http://jsoup.org/

[2] Hello World Examples: http://www.mkyong.com/java/jsoup-html-parser-hello-world-examples/

[3] Download images (Stackoverflow Q & A): http://stackoverflow.com/questions/5882005/how-to-download-image-from-any-web-page-in-java

Using webcam in C# applications

Introduction

During our final year research project I had to find on things associated with visual images. While searching for those things I came across ways which can use web cam with a c# application. While doing digging in this area I came across various solutions, but initial ones didn’t took my interest. After going across some links I found that the most interesting thing which can use for this purpose is the AForge.NET framework. Therefore I thought of keeping a bookmark on that for anyone who is interested.

AForge.NET framework allows users to do many interesting things easily. It has already contains waste number of examples from just taking an image from a web cam to image recognition and neural networks. In most cases this AForge.NET framework will not require other libraries except basic c# libraries. In references you can find some cool things done using Aforge.NET framework.

In this post I’m going to explain on very basic application, which allows you to start & stop displaying web cam input and store a captured image. In a previous post I have done the same thing in Java using JMF framework.

Design of the application

First of all let’s have a look at the design of the application. It has two buttons, a pictureBox, and a comboBox on a Form. Their names are given as follows.

Application Design
Application Design

After the design you can start coding. But before start coding you need to download the Aforge.NET framework and add the following references to the project.

  • AForge.Video.dll
  • AForge.Video.DirectShow.dll

These files are reside inside Release folder of the downloaded folder. You can add those references through Solution Explorer window in Visual Studio. Thereafter you can add those references with ‘using’ keyword to any .cs file when required.

Coding

Now let me give a brief explanation on the coding. It initially takes the all video input devices when loading and display as a list in combo box. That is done through getCamList() method. Thereafter depending on the selected device, input will be shown in the pictureBox1. Connection between device input and pictureBox is done through a FrameEventHandler connected to video source.

Other than that basic functionality, Capture button will save the current image inside pictureBox along with the time.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using AForge.Video.DirectShow;
using AForge.Video;
namespace WindowsFormsApplication1
{
 public partial class Form1 : Form
 {
 private bool DeviceExist = false;
 private FilterInfoCollection videoDevices;
 private VideoCaptureDevice videoSource = null;
public Form1()
 {
 InitializeComponent();
 }
private void Form1_Load(object sender, EventArgs e)
 {
 getCamList();
 }
private void getCamList()
 {
 try
 {
 videoDevices = new FilterInfoCollection(FilterCategory.VideoInputDevice);
 comboBox1.Items.Clear();
 if (videoDevices.Count == 0)
 throw new ApplicationException();
DeviceExist = true;
 foreach (FilterInfo device in videoDevices)
 {
 comboBox1.Items.Add(device.Name);
 }
 comboBox1.SelectedIndex = 0;
 }
 catch (ApplicationException)
 {
 DeviceExist = false;
 MessageBox.Show("No capture device on your system");
 }
 }
private void start_Click(object sender, EventArgs e)
 {
 if (start.Text == &quot;Start&quot;)
 {
 if (DeviceExist)
 {
 videoSource = new VideoCaptureDevice(videoDevices[comboBox1.SelectedIndex].MonikerString);
 videoSource.NewFrame += new NewFrameEventHandler(video_NewFrame);
 CloseVideoSource();
 videoSource.DesiredFrameSize = new Size(320, 240);
 videoSource.Start();
 start.Text = "Stop";
 }
 else
 {
  MessageBox.Show(&quot;Error: No Device selected.&quot;);
 }
 }
 else
 {
 if (videoSource.IsRunning)
 {
  CloseVideoSource();
  start.Text = "Start";
 }
 }
 }
//close the device safely
 private void CloseVideoSource()
 {
 if (!(videoSource == null))
 if (videoSource.IsRunning)
 {
 videoSource.SignalToStop();
 videoSource = null;
 }
 }
//eventhandler if new frame is ready
 private void video_NewFrame(object sender, NewFrameEventArgs eventArgs)
 {
 Bitmap img = (Bitmap)eventArgs.Frame.Clone();
 pictureBox1.Image = img;
 }
private void Form1_FormClosed(object sender, FormClosedEventArgs e)
 {
 CloseVideoSource();
 }
private void capture_Click(object sender, EventArgs e)
 {
  pictureBox1.Image.Save("Capture-" + DateTime.Now.ToString("HH-mm-ss tt") + ".jpg";);
 }
 }
}

The above code is the basis for any webcam application which uses Aforge.NET libraries. You are free to develop variety of other applications which using AForge.NET framework. Some of them may be object detecting systems, intelligent game applications, gesture recognizing systems, motion detecting systems, etc. Following is a snapshot of an object detecting application I created. Good luck with AForge.NET!

Simple Shape Detecting Application
Simple Shape Detecting Application

References

  1. AForge.NET framework official site – http://www.aforgenet.com/framework/
  2. AForge.NET samples – http://www.aforgenet.com/framework/samples/image_processing.html
  3. Simple webcam application – http://haryoktav.wordpress.com/2009/03/21/webcam-in-c-aforgenet/
  4. Shape Recognition application – http://leakingmemory.wordpress.com/2012/03/17/shape-recognition-using-c-and-aforge/
  5. How shape detector work – http://www.aforgenet.com/articles/shape_checker/
  6. Edge detection application – http://premsivakumar.wordpress.com/2010/12/13/edge-detection-using-c-and-aforge-net/
  7. Playing card recognition – http://www.codeproject.com/Articles/265354/Playing-Card-Recognition-Using-AForge-Net-Framewor
  8. Hand gesture recognition – http://www.codeproject.com/Articles/26280/Hands-Gesture-Recognition
  9. Recognition of handwritten digits – http://www.codeproject.com/Articles/143059/Neural-Network-for-Recognition-of-Handwritten-Digi
  10. Robotic samples – http://www.funnyrobotics.com/2010/12/brain-power-for-your-robot-aforgenet-c.html

How to create a custom window

Introduction

After ending of a terrible semester, I decided to start blogging again. During our projects in campus we had to create applications for various purposes, and I’ve used Window type forms like this in many times.

But I was looking for something different. Why can’t you create a form in a shape you wish? That was really difficult in past. You had to know the underlying technology about Window type form. But with improvements in technology, today it has become so easy. Here are two forms I’ve created.

So you may also interested in doing things like this. So let me show a quick guide to do it. For these forms I used Microsoft Expression Blend 3 for designing purposes and Microsoft Visual C# for giving logic.

First of all I’ll introduce two new terms that are going to use.

WPF– the Windows Presentation Foundation is Microsoft next generation UI framework to create applications with a rich user experience. It is part of the .NET framework 3.0 and higher. WPF combines application UIs, 2D graphics, 3D graphics, documents and multimedia into one single framework.

XAML-the Extensible Application Markup Language which specifies the appearance.

How to do it

1. Create a customized window.

Open Expression blend. Go to File -> New project and start a new WPF Application. Now Select “Window” from “Objects and Timeline” section. Then select “Properties” from the panel that appears on the right. Go to “Appearance” section in that panel and tick “Allows Transparency” property. Then in “Brushes” section, for “Background” select “No brush”.  That is the trick used to hide conventional window form.

Now you can create your own window using “Rectangle”, “Ellipse” tools provided in tool bar. Make sure your drawing is within the “Window” range. Else you can resize the “Window” size. Finally go to Projects -> Run Project to view result. I’m sure you may not happy as you get a static image in the middle of the screen. This will solve in next two sections.

2. Create a customized button

Draw the shape of the button using “Rectangle”, “Ellipse” tools. Right click on it and select “Make Into Control …”. From the window you get click “Button” and click “ok”. That’s it. This will turn your drawing in to a button.

Meanwhile remember that Blend also provides regular type of buttons that you see in many applications. This method is to make your drawing into a button.

3. Coding

This is the part which gives the life to our drawings.

First give a name to the button you created. Just type the name in “Name” property at “Properties” panel. Then click “Events” button. Give a name for event you want (in here Click event is named as “Clicked”) and press Enter key.

You will get a .cs file, which provides space to enter your C# logic code.

So those are the steps for a basic application. But there’s more you can do. Creativity is the limit. Hope you will do more creative UI’s and enjoy! 😀

PS: I haven’t found a way to do this type of stuff with Java. If someone know,  please share it here.

Resources

  1. Download Microsoft Expression Blend – http://www.microsoft.com/expression/products/Blend_Overview.aspx
  2. Kirupa tutorials – http://www.kirupa.com/blend_wpf/index.htm
  3. WPF tutorials – http://www.wpftutorial.net/Home.html
  4. DotNetCurry – http://www.dotnetcurry.com/ShowArticle.aspx?ID=348

Speaking and Listening Applications

Introduction

After a long period of silence, I decided to write my next blog post. I know that this is very interesting area and lots of new experiments going on. So in here I’m going to tell how to integrate text-to-speech and voice recognition capabilities to your application. This will help you to set your first step in this area.

This time I selected Visual C# as the language, because it provides very easy way to implement those. It is also essential to have .NET framework 3.0 or later version. With those implementing speaking /listening application is only a matter of few lines of code.

How to start

First open Visual C sharp and select a new project which allows building Console Application.

Create new project

For our purpose we have to add Speech API for our References. (By default it is not added)

For that Go to Solution Explorer, Right click on References and Click Add References.

Add Reference

From the Add reference window go to .NET tab, Click System.Speech and click OK.

Now the environment has settled. It’s time to code.

Text to speech application

First you need to add the following using statement.

using System.Speech.Synthesis;

Then add the following code where you want to implement the Text to speech capability.

SpeechSynthesizer synth = new SpeechSynthesizer();

synth.Speak("Hello from Student Guru!");

First it creates a SpeechSynthesizer object and then calls its Speak method. The text you want to read should be passed as the argument. But Speak method holds the program until its finish. This won’t create a program interactive. So instead Speak() you can use SpeakAsync(), which allows you to  execute your program while speaking is going on.

In addition to that SpeechSynthesizer alllow you to change voice using SelectVoice() method and save voice to .wav file using SetOutputToWaveFile() method.

Speech Recognition application

First you need to add

using System.Speech.Recognition;

Then add following code to a method you wish to do speech recognising,

SpeechRecognitionEngine recognitionEngine = new SpeechRecognitionEngine();

recognitionEngine.SetInputToDefaultAudioDevice();

recognitionEngine.LoadGrammar(new DictationGrammar());

RecognitionResult result = recognitionEngine.Recognize(new TimeSpan(0, 0, 20));

foreach (RecognizedWordUnit word in result.Words)

{

Console.Write("{0} ", word.Text);

}

After executing program, you have to speak to computer and it will detect your voice within first 20 seconds. If it fails to recognize any word it will show an error.

Note that this won’t give 100% accurate outputs, but this is for new comers.

In here I used DictationGrammer object to LoadGrammer, which is provided by windows desktop speech technology. But for special tasks you can build a new Grammer object using GrammerBuilder.

Recognition engine also allows you to read from a wave file and do the same.

References:

  1. Dr.dobbs web site:  http://drdobbs.com/windows/223101480?pgno=1
  2. StudentGuru: http://studentguru.gr/b/jupiter/archive/2010/02/01/speech-recognition-with-c-dictation-and-custom-grammar.aspx
  3. Microsoft Speech SDK: http://www.nextup.com/sapi5doc/Getting_Started.htm#Programmers_Guide