Using ADDRESS in Excel

As promised, today I am going to cover how to use the ADDRESS function in Excel.

First a word on Excel address format.  Excel's format, and generally any other Spreadsheet software worth its salt, for addresses (or references, whatever you like to call them) is of the following format.

'Some\Path\To\File[actualfile.xlsx]Worksheetname'!A14:A26

So that's a single quote, followed by the path to the file (if it is not currently open), followed by the file name enclosed by brackets, followed by a single quote, followed by an exclamation mark, followed by the cell or range being referenced.  All of this is pretty important when using the address formula, I'll cover why here in just a second, but first let's look at the address formula.

Syntax of ADDRESS
row_num	column_num	abs_num	a1	sheet_txt
This is the row number that you wish to reference.  This is the row relative row 1.  So a one here would mean row 1.	This is the column number that you wish to reference.  This is the column relative to column A.  So a one here would mean column A.	This is one of those, you have to remember it kind of values.  It dictates the type of reference to return.  The easiest way to remember it is how the formula is laid out.  Row before column.	This is a boolean that specifies if you want A1 style or R1C1 style returned to you.  Again, like INDIRECT, true means A1 style, false means R1C! style.  Default is true.	Basically, if you put some text here, then the address that will be returned will have an exclamation mark, automatically added and the text found here will be placed in front of the exclamation mark.

Here are the options for the abs_num parameter:

Value	Meaning
1	All absolute.  So basically the return will be something like $A$1.
2	Just row absolute.  So basically the return will be something like A$1.
3	Just column absolute.  So basically the return will be something like $A1.
4	All relative.  Return will be A1.

By default if you give no value for abs_num it will assume that you mean all absolute. So this all sounds good, let's take a look at an example:

	A	B	C
1	3	3
2
3	76	41	26
4
5
6	=ADDRESS(A1,B1)	=INDIRECT(A6)

The value of A6 here would be $C$3 and the value of B6 would be 26, since that is the value in $C$3.

As you can see from this example, we can now change the value of A1 or B1 to change the value that the formula in B6 will get.  This becomes very powerful if you want to create ranges that your end users can size up.  One can simply use =ADDRESS(A1,B1)&":"&ADDRESS(A2,B2) to get a range.  Now toss in a bit of SUM and let's see what happens.

	A	B	C
1	3	1
2	3	3
3	76	41	26
4
5
6	=ADDRESS(A1,B1)&":"&ADDRESS(A2,B2)	=SUM(INDIRECT(A6))

Here the value of A6 is $C$1:$C$3 and the value of B6 is 143, because that is the sum of the range of $C$1:$C$3.  I could have just reused the value in A1 to limit my sum to just a single row.  However, imagine what you can do with this.  Now you can build entry forms on your Excel sheets that allow the end user to type in (or using data validation, by a drop down list) the information that they are interested in, and it will go out and perform the needed formulas for them, without having to retype the formula or select a range.

Finally, if you want to get information from other places, not just on the sheet where the address formula is located, make sure you use the sheet_txt parameter.  You won't have to include the single quotes or the exclamation mark, but the rest you'll need to include if needed.  For example, if you are simply referencing another sheet in the workbook, you just need to give the sheet name.  However, if you are referencing a whole different file, you need to put in the whole shebang for the name.

Now that you know how to use this, you can use things like HYPERLINK, to create links to where the formula is getting its values, additionally, you can use the MATCH formula to change which row you want to get data from.  I'll show you a complete example of using MATCH and HYPERLINK with ADDRESS and INDIRECT next time.

Bringing it all together

Well, I got moved around at work and priorities got changed. Additionally, I've been sucked into the black hole that is Google+. SO I'm very sorry for the lack of posts for like the last forever now. Anyway, I have another Blog that deals mostly with SQL and what not, but I'm going to take that blog and this one and merge them together. See what happens. I'll most likely be changing the name of this blog as well to better reflect whatever it is that I start putting up here. So, here goes with the move! Wish me luck.

Too long...

So it has been too long since I last posted an update on blogger.

I will post again before too long.  Thing are dying down a bit at my work.  It hasn't given me much time to work on the TN5250 emulator.

I will say this.  I looked over the TN5250 Extended protocol while on site in another state not too long ago.  It's a pretty hefty spec.  So I may do extended down the road, but I'm pretty sure the first go round will not have extended support.

One thing I would like to do is allow some sort of scripting and reading from CSV files.  However, that's a pipe dream at the moment.  There's a lot that I need to work on in order to get the project into a useable state.  None of which I have time for.

Any who, here's look to my next post on Android TN5250.

More delay

I haven't posted in a while due to my job having a few demands for some migration work.  This has also taking a toll on the tn5250 emulator I'm writing for Android.

Additionally, I've been reading up on tn5250e stuff.  The protocol allows for things like menus and radio buttons to be implemented into the binary stream.  I hadn't considered adding tn5250e to my emulator and most likely I won't add it either.  Too much of a headache.

I'll be able to add small things to my code over the next few weeks, but I won't really get to do major coding until August.  Bummer...

I want to have my first usable release before I push the source to the open, but the emulator will be GPLv2.  I may make the libraries LGPLv2.  The first release may be a bit monolithic.

Been busy as of late.

19

So I wanted to cover real quickly the basic structure of the program that I will be using to watch socket packets for a TN5250 stream I will be connecting to.

/-----\         /----------\         /---------------\
| GUI |   <==>  | Activity |   <==>  | Packet reader |
\-----/         \----------/         \---------------/
                    /\
                    ||
                    ||  (passes back an open socket)
                    \/
                /----------------\
                | Socket Builder |
                \----------------/

So that's the basic structure. This isn't a TN5250 client as it is more a packet sniffer. Basically the socket builder is made it's own object just in case I need to change how a socket is made. The packet reader is the same reason, just in case I need to read the packets differently. I'll be writing this on Sunday and testing it on one of the open AS400 systems on the Internet.

At the same time I want to cover another wonderful point of telnet and TN5250, code pages. Over the years we've all become very accustom to a thing called UTF. Basically, this has become the agreed upon standard for encoding languages of the world. I won't cover how UTF works but it has greatly simplified how languages are encoded. However, AS400 systems talk in a very cryptic language call EBCDIC. So here's the deal. EBCDIC is eight bits (256 possible codepoints) when building a TN5250 terminal, one needs to be able to take the binary value and convert it to a UTF-8 character. How does that work? Let's go over a little example.

If you head over to the Wikipedia page for EBCDIC, you will see that the letters A through I are 0xC1 through 0xC9, in binary that would be 11000001 through 11001001. In decimal that would be 193 through 201. To have a conversion from that to UTF-8 one could build an array like so...

//blah blah init code

public static char [] cp037 = new char[256];
//...
//...
cp037[193] = '\u0041';          //Letter A
cp037[194] = '\u0042';          //Letter B
cp037[195] = '\u0043';          //Letter C
cp037[196] = '\u0044';          //Letter D
cp037[197] = '\u0045';          //Letter E
cp037[198] = '\u0046';          //Letter F
cp037[199] = '\u0047';          //Letter G
cp037[200] = '\u0048';          //Letter H
cp037[201] = '\u0049';          //Letter I

//Okay here comes the fun part of EBCDIC...  
//The alphabet isn't in numeric order so the 
//next character after I is soft hyphen...  
//We don't get to J until 209.

cp037[202] = '\u00AD';          //Soft Hyphen (look it up if you don't know)
cp037[203] = '\u00F4';          //Letter Ô
cp037[204] = '\u00F6';          //Letter Ö
cp037[205] = '\u00F2';          //Letter Ò
cp037[206] = '\u00F3';          //letter Ó
cp037[207] = '\u00F5';          //Letter Õ, also this is 0xCF, next is 0xD0
cp037[208] = '\u007D';          // the } symbol
cp037[209] = '\u004A';          //Finally, the Letter J...

Of course if you are really slick you can just define the array and init the values all in one go. Also, you might want to make the array final.

public static final char [] cp037 = { 
//192 other elements
'\u0041', '\u0042', '\u0043', '\u0044', '\u0045', '\u0046', '\u0047'
//You get the point hopefully.
};

Also I want to note one more hateful thing. You'll notice that I have called the array cp037... That's because EBCDIC is an 8-bit encoding, so there are multiple code pages to describe each language. In EBCDIC code page 037 there are enough characters to describe the languages of Australia, Brazil, Canada, New Zealand, Portugal, South Africa, and USA, except that 037 has no Euro support. Code Page 037 with the Euro symbol is known as code page 1140. The difference between 037 and 1140 is a single character. At code point 9F in EBCDIC in CP 037 is \u00A4 or ¤. At code point 9F in EBCDIC in CP 1140 is \u20AC or €. In other words the difference between the two in code is:

//Remember code point 9F = 159 in decimal.

cp037[159] = '\u00A4';          //The ¤ symbol
cp1140[159] = '\u20AC';         //The € symbol

Other then that, cp037 and cp1140 are exactly the same.

To actually implement every single code page in EBCDIC would be very difficult by myself. However, the nice thing is that there are several resources on the Internet that quote the IBM guidebook on the layout of each code page. This is nice because A) It's in digital format, B) I can copy and paste and reformat outside the IDE and then copy paste into the IDE when I've got it actually looking like code.

Anyway, I've beaten this EBCDIC to UTF-8 conversion thing to death. Yet, I have not answered why one must do this... Simple, because IBM hates you. Actually, Java speaks UTF-8 down to the core, but IBM speaks EBCDIC (which the story is funny because they made EBCDIC as a competitor to ASCII [guess who won]). So just like any program that has to speak to different encoding machines, you have to convert between the two. Usually one doesn't have to worry about this kind of stuff because some sort of library already exists to do all of the conversion without your knowledge. Well, that's exactly what we are doing. In order to have a TN5250 terminal we need to write a EBCDIC to UTF-8 converter library. Now you might wonder, why hasn't anyone already done this. Well a couple of people have, but they didn't make it into a library per se. So I can't use those pieces because those pieces don't stand on their own. In other words, the EBCDIC to UTF-8 converter is hard coded into the software that uses it, and thus cannot be separated without a lot of pain. Which then it becomes a "lesser of two evils" kind of deal.

Well I hope you all have enjoyed this little foray in encodings. Well, off to bed and tomorrow morning some Android socket programming for me!

Android Sockets

So I thought that socket programming was going to be difficult for the Android platform. Turns out, Android uses the standard java.net.Socket API. So your basic Java socket program also applies to Android. Now if you are a Java programmer, then you'll know that there are two different ways of using sockets in the standard library provided by the JRE. There are a ton of other socket APIs out there, but they are third party. The JRE only comes with two APIs for sockets.

1. The java.net.Socket API and related family
2. The NIO way of doing it.

The old socket way was invented to mimic Unix sockets. The NIO way is much more complex and I've only done a little bit of programming in the NIO way, so I'm relived that Android uses the old socket API. Enough beating around the bush, let me show you a very basic example of a client using java.net.Socket.

package lan.testing.SimpleClient;

import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.net.Socket;
import java.net.UnknownHostException;

import android.app.Activity;
import android.os.Bundle;
import android.view.View;
import android.widget.Button;
import android.widget.EditText;
import android.widget.TextView;

public class SimpleClient extends Activity {

  private EditText messageSend;
  private TextView messageRecv;

  /**Setup all your GUI stuff */
  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.main);
  
    messageSend = (EditText)findViewById(R.id.messagesend);
    messageRecv = (TextView)findViewById(R.id.messagerecv);
    Button buttonSend = (Button)findViewById(R.id.send);
    buttonSend.setOnClickListener(buttonSendOnClickListener);
  }

  Button.OnClickListener buttonSendOnClickListener = new Button.OnClickListener(){

    @Override
    public void onClick(View arg0) {
      Socket socket = null;
      DataOutputStream dataOutputStream = null;
      DataInputStream dataInputStream = null;

      try {
 socket = new Socket("192.168.99.1", 8888);
 dataOutputStream = new DataOutputStream(socket.getOutputStream());
 dataInputStream = new DataInputStream(socket.getInputStream());
 dataOutputStream.writeUTF(messageSend.getText().toString());
 messageRecv.setText(dataInputStream.readUTF());
      } 
      catch (UnknownHostException e) {
 e.printStackTrace();
      }
      catch (IOException e) {
 e.printStackTrace();
      }
      finally{
 if (socket != null){
   try {
     socket.close();
   }
   catch (IOException e) {
     e.printStackTrace();
   }
 }
  
 if (dataOutputStream != null){
   try {
     dataOutputStream.close();
   } 
   catch (IOException e) {
     e.printStackTrace();
   }
 }

 if (dataInputStream != null){
   try {
     dataInputStream.close();
   }
   catch (IOException e) {
     e.printStackTrace();
   }
 }
      }
    }
  };
}

That there is a basic client application that will send a message to 192.168.99.1 on port 8888 and then receive a reply back. The message to send is in a text edit box and the message received is placed in a text view widget. If you've ever taken a college course on Java Sockets then Android Sockets will make you feel right at home. Very little if anything is different between the two.

Next go round, I will create a socket and hold a telnet negotiation conversation with the server. Sorry if post seem to be stretched out. Company has me working on a tool for working with our vast store of XML documents, fun on the bun. Also, I'm still reading the RFC on the TN5250 stream format.

Foray into a telnet conversation

So here is a little bit of a recorded conversation that someone from the Internet had with a telnet server:

server:  IAC DO TERMINAL_TYPE
client:  IAC WILL TERMINAL_TYPE

server:  IAC DO OPT_END_OF_RECORD
client:  IAC WILL OPT_END_OF_RECORD

server:  IAC DO TRANSMIT_BINARY
client:  IAC WILL TRANSMIT_BINARY

server:  IAC DO TIMING_MARK
client:  IAC WONT TIMING_MARK

server:  IAC DO NEW_ENVIRONMENT
client:  IAC WILL NEW_ENVIRONMENT

server:  IAC SB TERMINAL_TYPE
client:  IAC SB TERMINAL_TYPE QUAL_IS IBM-3179-2 IAC SE

Or in hexadecimal:

server:  FF FD 18
client:  FF FB 18

sever:  FF FD 19
client:  FF FB 19

server:  FF FD 00
client:  FF FB 00

server:  FF FD 06
client:  FF FC 06

server:  FF FD 27
client:  FF FB 27

server:  FF FA 18
client:  FF FA 18 00 49 42 4D 2D 33 31 37 39 2D 32 FF F0

As you can see there isn't really a rhyme or reason to when the questions will be asked in a conversation. The server asked if the user would negotiate a terminal type first and then started that process after asking a couple of other questions. Also, it asked if the user would use timing marks (RFC 860: http://tools.ietf.org/html/rfc860) and the user said that they would not do so. Saying that you won't do something when the server asks you to, could end the connection. However, as you can see here, the server takes note of the WONT TIMING_MARK and apparently can deal with this refusal since the conversation continues. Not every server will be as forgiving.

This brings up what I'll need to watch for when building a tn5250 client during the start up of the socket. Basically, I'll need to watch for certain questions and respond by building answers. The simplest way of do this would be a switch statement. I have heard from some forums, that IBM iSeries systems seem to toss out extra IAC DO for no reasons. Now the thing is this, it will toss out IAC DO, but there won't be anything that follows up, so the stream could be IAC DO IAC DO TERMINAL_TYPE. The thing will be to check to see if this extra IAC DO exists in the stream and dump it.

With that said, I think the next step is to stop look at the telnet stream for a second and begin working on some example Android sockets. Next post will be a basic Android socket program. I am going to try to build a Java and Android base library (aka, a library that will work both in Java and Android) as the final result, but for now let's focus just on Android before we go back and try to make it platform agnostic.

Interesting things about Telnet. So the start of any foray into Telnet begins with the IETF RFC 854 (http://www.ietf.org/rfc/rfc854.txt). There you can get the gist of how the protocol works. Basically it covers what is know as the Network Virtual Terminal (NVT). This is basically the version of telnet that has zero additional features turned on. In other words, all optional features are turned off for NVT. That's not to say that NVT has no features, it just that all features that are in NVT are required to technically have a telnet client. A telnet client implements at least every feature that NVT supports. Telnet clients also may negotiate what features will or will not be used by an exchange of DO, DON'T, WILL, WON'T. All commands start with the IAC code (Interpret As Command). This is octal /377 or hex 0xFF. For example, most modern clients use full-duplex communication now-a-days. So one of the first things exchanged by telnet clients is IAC WILL SUPPRESS-GO-AHEAD or in hex 0xFF 0xF1 0x03, octal /377 /361 /003 (because for some reason octal is the what everything seems to be expressed for telnet.) Anyway, that whole SUPPRESS-GO-AHEAD function is not defined by IETF RFC 854, it is defined by RFC 858! Now we reach the problem I'm currently having with telnet. Going through and trying to find all of these dang RFCs that define telnet so that I have a modern client... Of course, once I have then all in hand, I'll still need to figure out which ones are relevant to tn5250... At the same time, I'm brushing up on Android socket programming. Believe it or not, the SDK documentation is pretty good. I'll still do a couple of dummy echo server/client programs to get a feel for socket programming in Android. The nice thing is that I have two tablets with Ice Cream Sandwich on them, so I'll be able to play with socket programming on them. One the client and one the server. Also, I'm not going to be targeting anything below 4.0. If the client runs on Android pre-ICS, then it is by pure dumb luck. Finally, I'm going to have to find a better code syntax highlighter for this blog before I start putting code up here. Cheers!

Whoa! I'm not dead I swear!

I've been mostly silent on Blogger.  I've just not had enough time with all of the projects that get tossed at me at work...  Well since I was last posting to this site I have now come across a new pet project of mine.  TELNET!  Now I know, why would anyone want to study a protocol that has been replaced by better protocols?  The answer is to develop a tn5250 terminal for Android that is awesome!  I currently, don't know of any tn5250 terminal for Android so it would be cool to have one up on the market.  Obviously, it will be free of charge.  I just cannot imagine charging people for something that I hope to be really useful!

Right now I'm posting this to my blogger account via an Android app.  So as you can see, I'm really liking the Android platform, but I think that there is a huge lacking of the open source movement on Android.  Some apps on the market will have links to the source code, but not a lot.  So I'm really committed to getting this tn5250 emulator up on the market and finding somewhere to host the source code.

Okay so now, I bet you are thinking.  How hard can a telnet client be?  If you are thinking that it is just a text based protocol, then you'd be dead wrong.  There are indeed special control characters in the telnet protocol and there is even a binary mode!  Basic telnet doesn't use any of these, but tn5250 takes advantage of everything plus some that telnet has to offer.  So this should be interesting.

Expect over the coming months, for me to paste my musings about telnet here.

PS:  Since I'll be working in Java, I'll look at making a nice little tn5250 buffer reader targeted at people who want to screen scrap tn5250 streams.  No promise because I really want to focus on the tn5250 emulator for Android.

Cheers!

Wow I haven't updated this blog i quite some time! Well. Not much has changed at work. However we have added Tomcat 6 to our mix but we are currently looking at options to move to Tomcat 7, however, it may be a while be fore we take the plunge. Anyway. I'll post some more snippets of code here. Maybe eventually I'll actually get a series going! Cheers everyone!

Grab the software you need.

Here's a short list of some of the things that you will need to have to follow along with me.

An actual JDK:

Okay in order to develop with Java you need a Java Development Kit.

I usually run with OpenJDK but you can always choose the Sun JDK. If you go with OpenJDK get version 6. Simply because that's all I'm going to cover for now. At some time in the near future ditch 6 for 7 but for now stick with 6, it is more tested and has fewer bugs in the JVM.

A Java EE server:

You'll need one of these in order to run your crap, er, programs. I like JBoss and I usually prefer it for EE 5 development (RedHat just released a EE 6 server but I haven't had time to test drive it). However, here I'm going to be using Glassfish v 3.0.1. It is a EE 6 server, and version 2.1.1 is a very good EE 5 server, it's what we used at work for some time. I'm going to be using the Web Profile version as opposed to the full version. You can grab that if you want but very rarely do I venture into a need for RMI-IIOP (using the server to distribute logic to clients). Later down the road I may cover RMI but for now I'm just going to cover WebEE.

A database!!:

I like MySQL. PostgreSQL is nice too. I won't get into too many details, in my clique there are some very sided opinions about which is better than the other. Do not forget to pick up the JDBC driver from whoever you choose.

A JPA library:

Glassfish comes with EclipseLink which is an awesome JPA provider. It's also the one that I will be using here. Other than it is provided with Glassfish, I know a good deal of the JPA hints for this stack, so I'm really in my zone with this. If you went out and got JBoss, you'll need to go grab EclipseLink here.

An IDE:

Okay notepad is great and all but at some point you're going to need to ditch it because it sucks as a development tool. Basically at some point everything just blends together and you spend more time trying to parse your source code with your eyes or you spend more and more time with comments to keep everything in some sort of order.

At a bare minimum you need an editor that can do syntax highlighting, this prevents all that black and white running together. One that can do auto-indentation (saves you at least five or ten minutes every day per project), auto-brackets (saves at least two or three minutes per day), and has the terminal very close by is even better (like Kate or gedit.) A full blown-out IDE like Eclipse or NetBeans can shave off thirty minutes to an hour per day per project if you get down in there and learn to really use it.

I'm going to be using NetBeans here for most of the examples. One, because it was one of the first Java IDEs I ever used and two, it was the one my college professor told us to use. See this is why Microsoft is making a killing in our college's in the Untied States, most...(never mind getting off topic).

Go get Netbeans here.

---

That looks like a lot to go get. Just remember that you are setting up for whatever may come your way. Aside from having some sort of code repo software (like git or so) you're basically using the same tools that a professional would be using for this task.

Well I better get myself off to bed. Cheers!

POJO extends vs implements

In Java we have two way that we implement inheritance either one extends a super class or implements an interface.

In JavaEE frameworks you will see a lot of this and understanding the difference between the two is pretty important.

Interfaces in Java have no actual code to back up the Interface. In other words implementing an interface requires you to add in all the code for the specified members in the interface. So what's the big deal then with interfaces?! They don't save you time do they? Usually an interface provides a hook into a library or a function of the EE server, so usually an interface will hook one object into another or the server proper.

Extending an object, however, provides the base class' implementation if you're feeling a bit lazy. Usually you will extend base classes to make something more specific to your use case. Going from something like a data table to an Employee data table. In Java you only get to extend one super class, as opposed to interfaces (which you can implement as many as those you want). I remember coming from all my C++ to Java and being frustrated at this.

---

It is also important to know about generics, generics allow your class to apply to more than one data type. For example instead of extending data table into employee data table, one could make data table generic data table<?>. Then you could say data table<Employee> or data table<Student>. Generics are extremely helpful for making common web concepts, like pagination, thumb viewers, and so on. The reason being is that the functionality doesn't change based on data type (A picture swoosher does the same thing to PNG as it would to JPG).

Finally annotations play a huge role in EE development in Java. Annotations allow a developer to add meta-data to their objects. A server can then read in this meta-data and change how it will behave with the object. Annotations only change the user's behavior not the objects. Remember that!

---

So why the lesson in general Object Oriented Programming? I think some people loose sight of the basics of computer science and that usually tends to lead to bad implementations of concepts.

I remember one time a person creating an interface with the action method to create a step-by-step process. I went back an implemented as a linked list style reader. People could use annotations to guide the reader like @PreCondition, @CleanUp,and of course @Action.

Of course no one is perfect, so I wouldn't say that anything on this blog is the end all, be all for implementations.

New Year, New Look, New Goal!

So it's a new year and I've changed the look of my Blogger account!

---

So I've also decided to realign my goal for info on this blog. The old stuff

will stay but I'm going to be better as keeping to one clear point, following it

through and not get side track too much.

I think it will be better overall this way.

SO! With that let's begin with the first topic.

JavaEE a broad view of what it is...

JavaEE is a collection of many different Java components to create a platform for server programming using Java (VM not language).

Basically it allows you to write software that will run and/or be distributed by servers running a JVM with the needed libraries. It is these libraries that separate JavaSE and JavaEE.

The platform is currently at version six but remember that the EE platform is a collection of components, each with their own version number.

Here is a quick table of some of the components in JavaEE 6. Now this is not a complete list, but this will cover the basics.

Component	Version	Summary
Servlet	3.0	Provides a means for a Java Object to provide a web page
Java Server Pages (JSP)	2.2	Allows the creation of Servlets using HTML files and java code embedded in the HTML code (very much like PHP)
Java Server Faces (JSF)	2.0	Provides a standard MVC library to HTML pages
Enterprise JavaBeans (EJB)	3.1	Provides a means to encapsulate business logic and deploy that logic to clients
Java Persistence API (JPA)	2.0	Provides a standard persistence layer to databases and Object Relationship Mapping (ORM).

Again this is just a brief overview of all that is in JavaEE 6.

Next post will cover POJOs and Interfaces.

Cheers!

Screen Scraping and scripting AS400 with Java!!

If you are like me you've been pulling your hair out trying to mess with the TN5250 stream so that you can automate things from you Java applications.

Well stop messing with the protocol itself and let TN5250j do that for you.

Here's how to do it with Netbeans 6.9:

First get TN5250j from here.

Next you will need Netbeans, duh. Netbeans!

Now start a new Java application. On the next screen name the program TestApp.

Now make Main.java say the following:

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */

package testapp;

import java.net.UnknownHostException;
import org.tn5250j.Session5250;
import org.tn5250j.beans.ProtocolBean;
import org.tn5250j.framework.tn5250.Screen5250;
import org.tn5250j.framework.tn5250.ScreenPlanes;

public class Main {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) throws UnknownHostException, InterruptedException {
        ProtocolBean pb = new ProtocolBean("test", "123");
        pb.setHostName("192.168.1.2");
        Session5250 session = pb.getSession();
        pb.connect();
        Screen5250 screen = session.getScreen();
        char [] buffer = new char[1920];
        StringBuilder sb = new StringBuilder();
        Thread.sleep(3000L);
        screen.GetScreen(buffer, 1920, ScreenPlanes.PLANE_TEXT);
        String showme = new String(buffer);
        for(int i=0;i<showme.length();i+=80) {
            sb.append(showme.substring(i, i+80));
            sb.append("\n");
        }
        System.out.println(sb.toString());
        session.disconnect();
    }

}

Run your program and you should see the results in your output window.

Let me run through these lines. I'll have to break up a detail description of each line over the next few post.

ProtocolBean is a bean that the TN5250j exposes that allows programmers to use the bulk of TN5250j's api to control the TN5250 data stream. It wraps all the nasty stuff into one nice little bean. There is one catch to this. You cannot use the ProtocolBean inside of the AWT-Event Loop. So if you are doing a GUI then you'll need to start the session outside of the AWT-Event Loop.

The ProtocolBean has two constructors. I'm using the more basic one here. Using the basic ctor means that you have to call a couple of methods from ProtocolBean before starting up. I'll cover the other ctor later. Here we call the setHostName method which should not surprise you that it sets the IP address of the host.

Oh and by-the-by if you are wondering what the ctor for the protocol bean is doing... The first argument creates a config file that TN5250j will use and the second names the session. So if you've ever used the PCOMM ECL then you know all about this, if not, Sessions have two names an ID and a name. The ID runs from 1 to 65536 or so, in the order that you start them. The name is whatever you like to call stuff. Libraries, can access a client by either the ID... Look I'm getting off topic, we'll save all that for another day.

Okay next thing is the Session5250. We grab the session first and then connect. Long story short, you should always grab the session first and then call the connect method from the protocol bean.

Next is to grab the screen. Now I'm on a really slow connection so I added a Thread.sleep() to slow things down. I'll cover later how you can actually check if the connection is ready to be used.

First grab the screen into a Screnn5250 object.
Next we need to make a char[] it will act as a buffer for all the data that we will get form the TN5250 stream. Yes, this is how you must do it. No, there is no way around it. I've checked the source code for TN5250j.

Next I'm just going to print to stdout the actual screen. My screen defaults to 24x80. Most of the time this is what the stream will always be. However, at my company we have a system that does 27x132. You need to check to make sure that the system isn't requesting 27x132 because most of the time this will make the session end abnormally if you don't report that you can handle 27x132.

As you can see I use a for loop to add a new line at every 80 chars. This makes it look exactly like the terminal.

Oh yeah if you are wondering about that GetScreen method. The parameters are char[] that will act as the buffer. How large is the buffer? And from which plane should I get data from? I'll explain ScreenPlanes.PLANE_TEXT later. For now that's the one that you want.

Anyway...

So then I System.out.println() my StringBuilder.

Then I call the disconnect method from the session object.

You can download the source code for TN5250j and read all about the Session5250 and Screen5250 classes.

I've got to go for now!

Oh and...

Just wanted to let you all know that the company Glassfish server was moved to version 3.0.1. We had no problems with 2.1 per se. We moved to 3.0.1 because we got clustering working, JNLP working, and we really needed some of the features of 3.0.1. Don't know which ones but the 3.0.1 was the one that we all finally settled on.

I was pushing JBoss but they would like to see 6.0 out of RC before going down that road.

Been gone for a while.

Well it's sad but I've been a way for a bit, at least blogging.

As you may know, Oracle bought out Sun and pretty much every Java community that was started by Sun and was still under Sun's wings during the last days have all abandoned Oracle. I don't blame them.

So I heard from friends, "So are you going to continue to develop using Java?"

The answer is of course. The JVM and the Java platform is not entirely Oracle's, they do have leadership over the core but the community of Java projects still exist, just no longer listening to Oracle. Oracle is slowly loosing the Java community and with it the fresh ideas that the community has produced, such as JSF 2.0 and EJB 3!

In the end Oracle will be playing catch up and Java's core will suffer but the community which has driven Java development for the last ten years will still stay strong. It's a shame because ideas from the community were actually being placed into the core Java spec towards the end of Sun's days.

Java as a platform isn't dead, the Java community isn't dead, just our relationship with Oracle. It's a shame that the bottom dollar is the only thing that they seem to think about but it's their bloody company and they can run how they please.

So what about Oracle's contributions to the open source world?

If you head over to Oracle's website you'll see the contributions that Oracle has made to the OSS community...

If you review all of their contributions you will see that they all lead back to Oracle products. I see their contributions on the same level as the Visual Studio Express versions that Microsoft offers. The independents will all die a slow death, which is sad because bringing them back via an OSS fork is always a slow to start process. Just look at OpenOffice.org and now Libre Office.

This shouldn't come as a surprise to anyone. But I think that this will make the Java community tighter and closer knit. Yes it does toss Java back about five to seven years, community / corporate wise, but I think the OpenOffice people are showing us all what the Netbeans, MySQL, Glassfish people will all eventually come to do, break ties with a company that refuses to work with the community and instead does the equal of food dumping into the open source community.

I will start posting more, I swear this time! We'll start slow, maybe twice a week?

Throbber for Java

Have you ever wanted to do a please wait for a Java application?  Did you want a little spinny logo to animate while they waited?

As Professor Fransworth would say, "Good news everybody!"

Let's start out with a preview of the graphic that I'll be using.


Now this is actually nine different images, I've just haphazardly put them into one image here so you can see what I'm doing. I made these images in about five minutes using Inkscape but you can use whatever. You do need to export the images into a format that Java understands, I usually default to png.

I've named the rotor images rotor1.png through rotor8.png and the final image (the check mark) as rotord.png. The check mark is optional and the code below is clear cut enough to be able to remove the check mark. At any rate I'm placing the images into a package called sample.img

Now here's what we need to do. We need to create a custom JFrame (of Swing fame) that will be our spinning logo. IF you are using Netbeans, DO NOT USE NEW JFrame CLASS. If you are using Netbeans select Create New Java class. This gives you a no frills attached blank document with a shell of a Java class to begin with. If you do use the JFrame class in Netbeans it adds an XML document behind the scenes that allows you to use the GUI-fied editor, which will only serve to get in your way, in this case at least.

So let's build our customer JFrame.

package sample.ui;

import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.Image;
import java.awt.Toolkit;
import javax.swing.JPanel;

/**
 *
 * @author John Doe
 */
public class RotorPanel extends JPanel {

    private Image[] rotorImg;
    private Image rotorComplete;
    private boolean done;
    private int current = 0;
    private final int max_image = 7;

    public RotorPanel() {
        this.rotorImg = new Image[max_image+1];
        for(int i=0; i<max_image+1; i++)
            this.rotorImg[i] = Toolkit.getDefaultToolkit().createImage(getClass().getResource("/sample/img/rotor" + (i+1) + ".png"));
        this.rotorComplete = Toolkit.getDefaultToolkit().createImage(getClass().getResource("/sample/img/rotord.png"));
        this.done = false;
    }

    @Override
    public void paint(Graphics g) {
        super.paint(g);
        Graphics2D g2 = (Graphics2D) g;
        if(!done)
            g2.drawImage(this.rotorImg[this.current], 0, 0, this);
        else
            g2.drawImage(this.rotorComplete, 0, 0, this);
    }

    public void increment() {
        this.current++;
        if(this.current>this.max_image)
            this.current = 0;
    }

    public void setDone(boolean done) {
        this.done = done;
    }

    public boolean isDone() {
        return done;
    }

}

As you can see it is pretty straight forward in approach. The constructor loads the images into an array and sets the done flag to false. The increment method moves us through the images. We've overridden the paint method to draw the image.

Now let's use our JPanel. If you are using Netbeans, go ahead an use the JDialog template, that's what I'll be covering here. In Netbeans add a standard JPanel from the palette and then in the properties window click on the code tab. In the custom creation code add "new RotorPanel()". This replaces the new javax.swing.JPanel that is found in the protected area (which is controlled by the XML file behind the scenes) with this code, which is what we want.

Next in the constructor code right after the initComponents call add the following code.

        this.t = new Thread() {
            @Override
            public void run() {
                RotorPanel p = (RotorPanel) PleaseWaitDialog.this.jPanel1;
                while(!p.isDone()) {
                    p.increment();
                    p.repaint();
                    try { Thread.sleep(65); }
                    catch (InterruptedException ex) {}
                }
                p.repaint();
            }
        };

This creates a thread that will call the increment method to rotate the image and then the repaint method which will update our image. The Thread.sleep(65) call is to sleep 65 ms and then repaint the image, this roughly controls the speed of the animation. Notice that the loop checks the done flag. When you are done and want this thread to die, simply call ((RotorPanel) this.jPanel1).setDone(true);

In 65 ms or less the thread will die because we exit the while loop and the run method of the thread returns. Of course this doesn't mean that you need to stop displaying something, just that you will no longer be rotating the image, hence the check mark image. The actual image doesn't go away until you kill the JDialog.

Pretty simple animation. If you want better control over animation and what-not, I would suggest looking into the timing framework that is located at https://timingframework.dev.java.net/. This framework will allow you have a better control over animating something. However, this is a simple little snippit of code that you can use for simple animations.

Cheers!

Powered by ScribeFire.

Websockets are cool!

I won't go into a lot of detail today, but I tried out serving up some PHP with websockets enabled. Websockets are way cool... I won't go to the end of saying, "OMG!! This will revolutionize the Internetz!!11!!" But I must say that this is will be a big game changer in terms of Internet applications. Combinded with HTML5, web sockets will be the new preferred method for messaging between server and client as opposed to AJAX and similar push technologies.


The best part of Web Sockets is the simple approach they take to passing messages between client and server combined with just how powerful they truly are. If you have ever used callback handlers then web sockets will be a very easy topic for you. You simply register what the web page should be listening for. You web page emits messages to the server and the server emits messages back. On each message received the web page checks what has been registered and the action associated with it.


This has the upshoot of being a write the handler and forget it approach which usually is a disservice to an API, but found that it serves JavaScript quite well.


The only problem will be getting frameworks to start jumping ship from AJAX to Websockets. XML can still be passed using Websockets and most likely that's the way most people will go, but any kind of data can be sent and that includes binary data which tends to be faster and more compact than XML. Most AJAX platforms already present their API as a series of callbacks for people to use in their web pages so Webdevs are already in the right thinking for Websockets.


Another barrier to websockets is the lack of servers that have support for it. The PHP version is still in beta and Apache feels that third parties will fill in the void for them. Jetty (Java server) and a few third party SilverLight stacks provide production grade Websockets at the current moment. The final barrier is the fact that the Websocket standard hasn't been approved as final by IETF, in fact it's still draft. So everything in the standard could change in a moments notice. That alone could keep vendors from adding it to their server. Browser support is Chrome, Opera, and Firefox 4. Basically all the next gen browsers, except our favorite browser to hate IE (including the latest IE 9 build).


All that said Websockets provide a new method of communicating with clients that out matches all currently existing technology and at some point full duplex communications is going to come to all browser and servers, be it websockets or something else down the road. Most likely Websockets will be the winner the biggest question will be in what shape as the standard could change as we all wait for it to become a final standard.

Using Cairo with Gtkmm

Okay we should be all familiar with the Cairo API. If this is the first time you've ever heard of Cairo outside of a reference to Egypt then let me give you a quick peek at Cairo.



Cairo is pretty much the main library used for drawing anything with vectors. It is used everywhere, sort of like SQLite. The website is http://cairographics.org/



In the GNOME trifecta of things, Cairo provides the 2D drawing API, whereas Pango provides the Text Rendering API, and Gtk-Glib provide the toolkit/glue for all the above.



The neat thing about Cairo is that output is not limited to the screen. Backends for Cairo can be written for anything, so things drawn in Cairo can be outputted to an SVG file, OpenGL commands, a printer, or to a Pixbuf for use in a very popular web browser (Firefox anyone).



So today I'm going to use Cairomm (C++ bindings for Cairo) to do some basic drawing. Here is our header file justin_draw.h that defines our DrawingArea.

#ifndef SAMPLE04_JUSTIN
#define SAMPLE04_JUSTIN

#include <gtkmm/drawingarea.h>

class JustinDraw : public Gtk::DrawingArea
{

public:
	JustinDraw();
	virtual ~JustinDraw();
	
protected:
	virtual bool on_expose_event(GdkEventExpose* event);

};

#endif

The only thing of note here is that we are going to define an on_expose_event. This is important because this is the method that is called when the window needs to draw itself. Either because the Window is new, or we've uncovered the window an exposed a part of it that was under another window.




Now let's implement our Gtk::DrawingArea with our justin_draw.cc file:

#include "justin_draw.h"
#include <cairomm/context.h>

JustinDraw::JustinDraw() {}

JustinDraw::~JustinDraw() {}

bool JustinDraw::on_expose_event(GdkEventExpose* event) 
{
	Glib::RefPtr<Gdk::Window> window = get_window();
	
	if (window)
	{
		Gtk::Allocation a = get_allocation();
		const int w = a.get_width();
		const int h = a.get_height();
		
		int xc, yc;
		xc = w/2;
		yc = h/2;
		
		Cairo::RefPtr<Cairo::Context> cr = window->create_cairo_context();
		cr->set_line_width(10.0);
		
		cr->rectangle(event->area.x, event->area.y, event->area.width, event->area.height);
		cr->clip();
		
		cr->set_source_rgb(0.8, 0.0, 0.0);
		cr->move_to(0,0);
		cr->line_to(xc,yc);
		cr->line_to(0,h);
		cr->move_to(xc,yc);
		cr->line_to(w,yc);
		cr->stroke();
	}
	
	return true;
}

Notice a couple of things here before we move on. Drawing an entire screen with Cairo should be done very rarely. Not to say Cairo is slow but there is no need to waste time on drawing. The cr->rectangle(...); cr->clip(); parts allow us to focus on only the part of the window that has changed. Pretty much all code that uses Cairo has something similar to this so it's a good idea to include something like this in you code before you actually do any drawing.




Also notice that we are using a Cairo::RefPtr. Yes Cairo has it's own type of RefPtr's that you must use to hold Cairo related things. You have no idea how much time you can waste using the wrong type of RefPtr.




Finally, our program that uses our new Gtk::DrawingArea. main.cc

#include "justin_draw.h"
#include <gtkmm/main.h>
#include <gtkmm/window.h>

int main(int argc, char** argv)
{
   Gtk::Main kit(argc, argv);

   Gtk::Window win;
   win.set_title("DrawingArea");

   JustinDraw area;
   win.add(area);
   area.show();

   Gtk::Main::run(win);

   return 0;
}

And that's all there is to it! You can hit up basically any manual on Cairo to find all the 2D API commands. It's basically a 1:1 translation between the C and C++ bindings so you shouldn't have any problems there.




Tomorrow I'm going to cover Making a throbbing image in Java, because I'm gotten two emails over the last four days asking how to do that. Till then, Cheers!