Speeding up git-svn for several users

2 Comments

Git-svn is a wonderful tool if you need to access an svn repo. Reading the docs, it discusses how slow the initial clone is and that you should do one clone, check it into git, and have others clone from you. I tried this, doing the following:


$ git svn clone svn://some/project
$ git push git+ssh://mygit.repo

All was good. I then attempted to clone this, and I saw the following:


$ git clone git+ssh://mygit.repo
$ git svn init svn://some/project
$ git svn rebase
Unable to determine upstream SVN information from working tree history

And worse then this, if I did a git-svn fetch it pulled down the entire repo again (4 hours). I am trying to find where in the code the issue is, but here is the workaroung. You need to add one file into the .git repo which points to the current commit’s hash.


# clone the git repo
$ git clone git+ssh://mygit.repo

# Attach it to the svn repo:
$ git svn init svn://some/project

# get the latest commit hash
$ cd mygit.repo
$ git log

# The latest hash will bet the first thing you see (commit [HASH])
# Copy it, and add it as a single line to the git-svn file
$ vi .git/refs/remotes/git-svn

# Now you can fetch the latest from subversion, and begin working
$ git svn fetch
$ git svn rebase 

QMF#4: Ruby accessing EJBs running in JBoss

Leave a comment

This is the latest in a series of posts on using qmf consoles in various languages to connect to EJBs running in a JBoss 5 container. There hass been python, C#, and Java. Now, lets add ruby to the list. Steps for building and deploy the Java agent can be found in the Java article. I will assume you have a running agent, a running broker, the latest qpid code, and would like to use the rby console now.

The current ruby console uses some native ruby extensions, so you first need to build those. Make sure you have ruby and rake installed


cd qpid/qpid/ruby/
rake build
export RUBYLIB=[QPID_SOURCE_LOCATION]/qpid/ruby/lib:[QPID_SOURCE_LOCATION]/qpid/qpid/ruby/ext/sasl/ 

Assuming you have the latest version of the git example code cloned from github, you can now run the following:


cd qmfExample/rubyConsole
ruby qmfExample.rb

You will see the same method calls, and XML Schema generated as from the earlier languages. A very easy way to expose your java code to many different languages.

QMF # 3: Java to Java over QMF

1 Comment

Here is the next post in the series. Java to Java over QMF. Granted, this is a pretty dumb way to actually connect to an EJB.. since Java provides many ways to do it natively. But, it does show a different console technology accessing the same backend. As before, you can read up on QMF and then build these examples. I will republish the directions from the earlier post to limit the number of times you need to scroll down.

To get started, you will need the gcc tools, ant, and jboss5 installed. Next.. get the code:


# Get the latest QPID Code
svn co http://svn.apache.org/repos/asf/qpid/trunk
# Get my example Code:
git clone git://github.com/bkearney/qmfExample.git

Build the latest qpidd broker and java code


cd trunk/qpid/cpp
./bootstrap
./configure --prefix ~/qpidlocal
make install
cp ../java
ant

Once this is done, you should be able to launch the most current qpidd


export LD_LIBRARY_PATH=~/qpidlocal/lib
~/qpidlocal/sbin/qpidd

Now…. build the java agent which has the EJB we are going to expose


cd ~/qmfExample/javaAgent
# Edit the build.properties file to point to your local
# installation
ant install

Now that we have that… we will build the Java Console to talk to the agent. As with the C# console earlier, the Java example code has 2 versions. One is using the raw QMF Session interface, and is the most similar to the python code. This is located in the qmfExample/javaConsole/generic directory. The second has some classes which were generated off of the example schema. This code is located in qmfExample/javaConsole/generated. The same logic can be used to generate either one.


cd qmfExample/javaConsole/generic
# edit your build.properties and set the property qpid.home and jboss.home
ant build
ant run

VIOLA! You see the EJB being accessed, and the schema being created. This is the third technology to access the example… next week.. ruby.

Now.. C# to EJB over QMF

1 Comment

This is a follow up to the earlier post about accessing EJBs from python.. In this post, we will build the CSharp Console and use that to access the EJBs which are running on JBoss. This code was built and tested using Mono on Fedora 10. My hope is that someone can test out the console in a windows proper environment. CSharp to dotnet is pretty hard with all but the most vanilla WS technologies. This post shows how the Qpid Management Framework can bridge that gap.

First.. follow the directions from the earlier post to check out the latest qpid code from apache, and the latest qmf example code from github. I will assume you have the broker running, and that you have tested the python client to see that it is working.

Now, lets build the dotnet qpid code. You will need ant, mono, and nant installed.


cd qpid/qpid/dotnet/client-010
# Read the README.txt file if things go wrong
cd gentool
ant
cd ..
nant

If you are getting a nansty nunit error, do the following before running nant:


export MONO_PATH=/usr/lib/mono/nunit22/

Now, we will build the test code which will use the console to access the Java Server. The example code has 2 versions. One is using the raw QMF Session interface, and is the most similar to the python code. This is located in the qmfExample/cSharpConsole/generic directory. The second has some classes which were generated off of the example schema. This code is located in qmfExample/cSharpConsole/generic. The same logic can be used to generate either one.


cd qmfExample/cSharpConsole/generic
# edit your default.build and set the property qpid.dir
nant
# it is now built, lets run it
export MONO_PATH=$MONO_PATH:[QPID_DIR]/qpid/qpid/dotnet/client-010/bin/mono-2.0/debug/
mono ./bin/QmfExample.exe

And.. Bobs your uncle. Now we have 2 technologies (python and C#) which are historically not friendly to Java, accessing EJBs running on Jboss.

Accessing JBoss Session beans from python using QMF

5 Comments

My current project is looking at how to expose Enterprise Java Beans to other technologies. I am most concerned with C#, but any other languages would be cool. We looked at the WS-* standards (and CXF which is a great tool) but the different versions across the vendors provided difficult.

We then looked at QMF, which is an management framework written on top of the QPID messaging fabric. You can learn about QMF here. This post will walk you through an eaxmple of exposing an EJB in jboss over the QMF Bus. Gregory Mostizky is working on a deployer, so this should get easier.. but this is a good enough to criticize.. so I am posting it.

To get started, you will need the gcc tools, ant, and jboss5 installed. Next.. get the code:

# Get the latest QPID Code
svn co http://svn.apache.org/repos/asf/qpid/trunk
# Get my example Code:
git clone git://github.com/bkearney/qmfExample.git

Build the latest qpidd broker and java code

cd trunk/qpid/cpp
./bootstrap
./configure --prefix ~/qpidlocal
make install
cp ../java
ant

Once this is done, you should be able to launch the most current qpidd

export LD_LIBRARY_PATH=~/qpidlocal/lib
~/qpidlocal/sbin/qpidd

Now…. build the example code:

cd ~/qmfExample/javaAgent
# Edit the build.properties file to point to your local
# installation
ant install

You can now start up jboss, and run the example python script to access the bean

export PYTHONPATH=~/trunk/qpid/python
cd ~/qmfExample/pythonConsole
python ./qmfExample.py

Thats all there is. Python is now calling EJBs! Now… lets see how we did it. All the code below is in the qmfExample/javaAgent/src
directory. Lets first look at the Session Bean we created:

package qmf.example.ejb;

import java.util.ArrayList;
import java.util.List;
import javax.ejb.Stateless;
import org.apache.qpid.agent.annotations.QMFHide;
import org.apache.qpid.agent.annotations.QMFObject;
import qmf.example.BaseClass;
import qmf.example.DerivedClass;

@Stateless(name="ServiceBean")
@QMFObject(className="ExampleService", packageName="qmf.example.ejb")
public class ServiceClassBean implements ServiceClass
{

    public BaseClass getBase(String name, String description) {
        BaseClass bc = new BaseClass() ;
        bc.setName(name) ;
        bc.setDescription(description) ;
        return bc ;
    }

    public DerivedClass getDerived(String name, String description, int count) {
        DerivedClass dc = new DerivedClass() ;
        dc.setName(name) ;
        dc.setDescription(description) ;
        dc.setCount(count) ;
        return dc ;
    }

    public List findMany() {
        ArrayList list= new ArrayList() ;
        list.add(getBase("JarJar", "My Friend")) ;
        list.add(getDerived("Binks", "Not My Friend", 12)) ;
        ArrayList data = new ArrayList() ;
        data.add(11) ;
        data.add("Info") ;
        data.add(getBase("Wotto", "Also My Friend")) ;
        list.get(0).setStuff(data) ;
        return list ;
    }

    @QMFHide
    public void doHokeyPokey() {
        System.out.println("Turn yourself around") ;
    }
}

This is pretty normal except for 2 annotations. One, @QMFObject, gives
the name and package to expose. The second, @QMFHide, causes the method
to be ignored. The bean returns 2 classes, a base class and a derived
class. We have annotated the base class so that the marshalling code
knows about the derived class. This is a similar pattern to JAXB:

package qmf.example;

import java.util.ArrayList;

import org.apache.qpid.agent.annotations.QMFSeeAlso;

@QMFSeeAlso({DerivedClass.class})
public class BaseClass
{
.....

The final magic, is a beans.xml file which tells the JBoss microcontainer to start up the agent. Here is the code for that:

<deployment xmlns="urn:jboss:bean-deployer:2.0">    

	<bean name="QPIDConnection" class="org.apache.qpid.client.AMQConnection">
		<constructor>
			<parameter>amqp://guest:guest@/?brokerlist='tcp://localhost'</parameter>
		</constructor>
	</bean>

    <bean name="ServiceBean" class="org.apache.qpid.agent.ManagedEJB">
        <property name="name">ServiceBean</property>
        <property name="className">qmf.example.ejb.ServiceClassBean</property>
        <property name="jndiLocation">qmfExample/ServiceBean/local</property>
    </bean>	    

	<bean name="QMFAgent" class="org.apache.qpid.agent.Agent">
		<property name="label">agent</property>
		<property name="sessionTransacted">false</property>
		<property name="connection">
			<inject bean="QPIDConnection" />
		</property>
		<property name="managedObjects">
			<list elementClass="org.apache.qpid.agent.ManagedObject">
				<inject bean="ServiceBean" />
			</list>
		</property>
		<property name="registeredClasses">
			<list elementClass="java.lang.String">
			</list>
		</property>
	</bean>

</deployment>

As I said earlier, the deployer should make this easier. Specifically.. it will remove the need for the beans file. But.. this will work until the deployer is done.

Funny where you see error messages

Leave a comment

Saw this in my swanky hotel room in NYC over the holidays. It was the main page on the movie/game menu system. The good news, it made sure my kids did not buy any movies without me knowing it.

Funny M$ error window

Funny M$ error window

All and all it was a great trip. We posted photos of the trip at http://picasaweb.google.com/bryan.kearney/NYCSpringBreak.

Connecting JBoss CXF Web Service with a WCF Client

4 Comments

I have been trying for a few days to get a JBoss web service using reliable messaging and ws addressing to talk to an M$ client. There are lots of useful tuts on the topic, but nothing that showed all the bits which I needed to set up. I have it working, so I figured I would post the important bits for other folks to see.

I am using the following stacks:
JBoss 5.0.1 running on Fedora 10
Jbossws-CXF 3.1.0
Visual Studio 2008 running on Server 2003

The combination which seems to work is
Soap 1.1 or 1.2
Reliable Messaging 1.0 (February 2005 version)
Addressing 1.0 (August 2004 version)

My Service is a standard EJB using annotaitons to link to the handcrafted wsdl. It seems that you need to use handcrafted wsdl to do anything interesting. Here is the web service definition:

package com.redhat.vdc.backend;

import javax.ejb.EJB;
import javax.ejb.Stateless;
import javax.jws.WebMethod;
import javax.jws.WebParam;
import javax.jws.WebResult;
import javax.jws.WebService;
import javax.jws.soap.SOAPBinding;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.jboss.wsf.spi.annotation.WebContext;

import com.redhat.vdc.action.VdcActionParameters;
import com.redhat.vdc.action.VdcActionType;
import com.redhat.vdc.action.VdcReturnValue;
import com.redhat.vdc.common.utils.SessionContainer;
import com.redhat.vdc.common.utils.VdcException;

@WebService(serviceName = "WCFBackendBean", wsdlLocation = "META-INF/wsdl/com.redhat.vdc.backend.WCFBackendBean.wsdl")
@WebContext(contextRoot = "/backends", urlPattern = "wcf")
@SOAPBinding(style = SOAPBinding.Style.DOCUMENT)
@Stateless(name = "WCFBackend")
public class WCFBackendBean extends VdcAdapter {

	@EJB(beanName = "Backend")
	Backend backend;

	Log log = LogFactory.getLog(WCFBackendBean.class);

	@WebMethod
	@WebResult(name = "vdcReturnValue")
	public VdcReturnValue runAction(
			@WebParam(name = "actionType") VdcActionType type,
			@WebParam(name = "parameters") VdcActionParameters params) {

		try {
			this.initializeSession() ;
			this.incrementRequestCounter() ;
			log.debug("Request Number: " + SessionContainer.getInstance().getData("requestCount")) ;
			return backend.runAction(type, params);
		} catch (VdcException e) {
			throw e;
		} catch (Exception e) {
			throw new VdcException(e);
		} finally {
			resetSession() ;
		}
	}

}

The WSDl is located in the META-INF directory of the EJB jar. It defines the standards listed above:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<definitions targetNamespace="http://backend.vdc.redhat.com/"
	name="BareBackendBean" xmlns:tns="http://backend.vdc.redhat.com/"
	xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
    xmlns:soap11="http://schemas.xmlsoap.org/wsdl/soap/"
    xmlns:soap12="http://java.sun.com/xml/ns/jaxws/2003/05/soap/bindings/HTTP/"
    xmlns:soap12CXF="http://schemas.xmlsoap.org/wsdl/soap12/"
	xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy"
	xmlns:wspp="http://java.sun.com/xml/ns/wsit/policy"
	xmlns:wsrmp="http://schemas.xmlsoap.org/ws/2005/02/rm/policy"
	xmlns:wsrmp11="http://docs.oasis-open.org/ws-rx/wsrmp/200702"
	xmlns:wsrmp10ForJavaNative="http://schemas.xmlsoap.org/ws/2005/02/rm"
	xmlns:wsrmp10="http://schemas.xmlsoap.org/ws/2005/02/rm/policy"
	xmlns:wsaw="http://www.w3.org/2006/05/addressing/wsdl"
	xmlns="http://schemas.xmlsoap.org/wsdl/">
	<types>
		<xsd:schema>
			<xsd:import namespace="http://backend.vdc.redhat.com/"
				schemaLocation="WCFBackendBean_schema1.xsd" />
		</xsd:schema>
	</types>
	<message name="runAction">
		<part name="parameters" element="tns:runAction" />
	</message>
	<message name="runActionResponse">
		<part name="parameters" element="tns:runActionResponse" />
	</message>
    <portType name="BareBackendBean">
        <operation name="runAction">
            <input
                wsaw:Action="http://backend.vdc.redhat.com/BareBackendBean/runActionRequest"
                message="tns:runAction">
            </input>
            <output message="tns:runActionResponse">
            </output>
        </operation>
    </portType>
	<binding name="BareBackendBeanPortBinding" type="tns:BareBackendBean">
		<wsaw:UsingAddressing />
		<soap:binding transport="http://schemas.xmlsoap.org/soap/http"
			style="document" />
		<operation name="runAction">
			<soap:operation soapAction="" />
			<input>
				<soap:body use="literal" />
			</input>
			<output>
				<soap:body use="literal" />
			</output>
		</operation>
	</binding>
	<service name="BareBackendBean">
		<port name="BareBackendBeanPort" binding="tns:BareBackendBeanPortBinding">
			<soap:address location="http://127.0.0.1:8080/backends/bare" />
		</port>
	</service>
</definitions>

Also in the META-INF directory is a jboss-cxf.xml file which has the following contents. Again, this references the versions of the WS standards listed above:

<beans xmlns='http://www.springframework.org/schema/beans'
	xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
	xmlns:beans='http://www.springframework.org/schema/beans'
	xmlns:jaxws='http://cxf.apache.org/jaxws'
	xmlns:wsp='http://www.w3.org/2006/07/ws-policy'
	xmlns:p='http://cxf.apache.org/policy'
	xsi:schemaLocation='http://www.springframework.org/schema/beans

http://www.springframework.org/schema/beans/spring-beans-2.0.xsd

http://cxf.apache.org/policy

http://cxf.apache.org/schemas/policy.xsd

http://www.w3.org/2006/07/ws-policy

http://www.w3.org/2006/11/ws-policy.xsd

http://cxf.apache.org/jaxws

http://cxf.apache.org/schemas/jaxws.xsd'>

    <bean id="logInbound" class="org.apache.cxf.interceptor.LoggingInInterceptor"/>
    <bean id="logOutbound" class="org.apache.cxf.interceptor.LoggingOutInterceptor"/>

	<wsp:Policy wsu:Id="wsrm10policy"
		xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
		<wswa:UsingAddressing xmlns:wswa="http://www.w3.org/2006/05/addressing/wsdl" />
		<wsrmp:RMAssertion xmlns:wsrmp="http://schemas.xmlsoap.org/ws/2005/02/rm/policy" >
            		<wsrmp:AcknowledgementInterval Milliseconds="1000"/>
		</wsrmp:RMAssertion>
	</wsp:Policy>

	<jaxws:endpoint id='WCFBackend'
		implementor='com.redhat.vdc.backend.WCFBackendBean'>
		<jaxws:invoker>
			<bean class='org.jboss.wsf.stack.cxf.InvokerJSE' />
		</jaxws:invoker>
		<jaxws:features>
			<p:policies namespace="http://www.w3.org/2006/07/ws-policy">
				<wsp:PolicyReference URI="#wsrm10policy"
					xmlns:wsp="http://www.w3.org/2006/07/ws-policy" />
			</p:policies>
		</jaxws:features>
		<jaxws:outInterceptors>
		    <ref bean="logOutbound"/>
		</jaxws:outInterceptors>
		<jaxws:inInterceptors>
		    <ref bean="logInbound"/>
		</jaxws:inInterceptors>
	</jaxws:endpoint>

</beans>

On the client side, I used the visual studio tool to create a reference to th web service. By default, I got a basicHttpBindings in the app.config file. The voodoo was to replace the basic binding with a custom binding that references the same WS standards. Here is the file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <system.serviceModel>
        <bindings>
            <customBinding>
                <binding name="WCFBackendBeanPortBinding1">
                    <textMessageEncoding maxReadPoolSize="64" maxWritePoolSize="16"
                        messageVersion="Soap11WSAddressingAugust2004" writeEncoding="utf-8">
                        <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
                            maxBytesPerRead="4096" maxNameTableCharCount="16384" />
                    </textMessageEncoding>
                  <reliableSession ordered="true" reliableMessagingVersion="WSReliableMessagingFebruary2005" flowControlEnabled="true" inactivityTimeout="00:15:00" />
                    <httpTransport manualAddressing="false" maxBufferPoolSize="524288"
                        maxReceivedMessageSize="65536" allowCookies="true" authenticationScheme="Anonymous"
                        bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
                        keepAliveEnabled="true" maxBufferSize="65536" proxyAuthenticationScheme="Anonymous"
                        realm="" transferMode="Buffered" unsafeConnectionNtlmAuthentication="false"
                        useDefaultWebProxy="true" />
                </binding>
            </customBinding>
        </bindings>
        <client>
            <endpoint address="http://hattrick.usersys.redhat.com:8080/backends/wcf"
                binding="customBinding" bindingConfiguration="WCFBackendBeanPortBinding1"
                contract="ServiceReferenceWCF.WCFBackendBean" name="WCFBackendBeanPort1" />
        </client>
    </system.serviceModel>
</configuration>

Deploy the ejb, compile the front end, and it works. The same endpoint can be use by a CXF client, so there is good interoperability between java and C#.

Newer Entries