Voice & Video Streaming, Whiteboard and Desktop Sharing Available in Openfire

Installing Red5 plugin to my Openfire server and enabling it was a breeze using the Openfire  web control console.

Now, I am reading about Real Time Messaging Protocol, and ActiveX controls to embed Flash Media into .Net forms (axShockwaveFlash COM component that you can add to toolbox in Visual Studio).

The Jingle implementation that exists in Spark supports voice chat without Flash. At least I can port that to my C# forms client implementation. However there seems to be a long delay. It may be due to my underpowered test server and hopefully improved when I start using our real server. If the delay is due to bandwidth, using better codecs to compress might solve the problem.

I have found out that ffmpeg supports flv and swf, and there are ways to use ffmpeg inside .net: calling ffmpeg.exe, using ActiveX control, external ActionScript API for C# (tutorial uses an instant messaging application, so should be useful)
http://help.adobe.com/en_US/ActionScript/3.0_ProgrammingAS3/WS5b3ccc516d4fbf351e63e3d118a9b90204-7cb0.html, http://help.adobe.com/en_US/ActionScript/3.0_ProgrammingAS3/WS5b3ccc516d4fbf351e63e3d118a9b8fc4e-8000.html [Code and Samples]

Resources:
Compiling Spark for Dummies:
http://www.igniterealtime.org/community/docs/DOC-1521

Spark client secure audio/video with Red5 plugin:
http://www.igniterealtime.org/community/docs/DOC-1633

Red5 Plugin Readme:
http://www.igniterealtime.org/community/docs/DOC-1518

Red5 Linux Setup:
http://www.igniterealtime.org/community/docs/DOC-1520

Simple C# and Flash interop:
http://bojordan.com/log/?p=269

Note:

Need to check DirectShow [or this? which is latest documentation] and OSMF [tutorial] as well.

F-IN-BOX is a developer’s library (for $300) to enhance Macromedia Flash Player ActiveX features. It does not use its own engine to display movies but provide a wrapper around official swflash.ocx/flash.ocx code instead. Thus it is possible to avoid certain Macromedia Flash Player ActiveX limitations.

Media Streaming Struggles

Finally, I have our chat server (openfire by JiveSoftware) running, my client connecting to it successfully (using jabber-net C# implementation of an XMPP API). Now I need to figure out how to stream video and audio.

XMPP has Jingle extension for negotiating streaming parameters (format, ports etc.). However, I haven’t found a .Net implementation. I have used JiveSoftware’s Spark client to connect to my server and had a successful voice chat which means Spark has implementation of Jingle. However Spark is implemented in Java, while I had implemented my client in C#. Now the question is to port my client to Java or to extract and port their Jingle library to C#…

Another good news is Openfire supports Flash Video and Audio streaming via a plugin implementing Adobe Flash Media Server (through reverse engineering) called red5. This might come handy when I want to support video as well.

Jingle only negotiates the stream parameters. I have to work on what I can use for compressing and decompressing media. Jingle doesn’t handle such things. Microsoft has a DirectShow API to perform various operations with media files or streams. (Media Foundation is replacing it, but Wikipedia says it won’t support Windows XP, which is still has around %60 of the OS market share.)

So currently I am researching about DirectShow and Flash based Media streaming APIs to incorporate into my client.

How to Get the Name of an Object’s Type (Class) in C#

string name = instanceVariable.GetType().Name;

“Indexed Properties” of a Class in C#

In Jabber-net library, which I am using for my client, I saw this piece of code, which basically stores an array of type-unsafe properties. It is a useful idiom to have in mind for C# developers.

/// <summary>

/// Sets or retrieves a connection property.

/// You have to know the type of the property based on the name.

/// For example, PORT is an integer.

/// </summary>

/// <param name=”prop”>The property to get or set.</param>

/// <returns></returns>

public object this[string prop]

{

get

{

if (!m_properties.Contains(prop))

return null;

return m_properties[prop];

}

set

{

m_properties[prop] = value;

if (PropertyChanged != null)

{

PropertyChanged(this, new PropertyChangedEventArgs(prop));

}

}

}

_________________________________

Used in this way (if stream is of this class):

stream[(string)parent.Tag] = GetControlValue(parent);

References:

While reading more about this, I learned it is called “Indexed Properties”.

http://msdn.microsoft.com/en-us/library/aa288464(VS.71).aspx

For preliminary on Indexers (like operator [] in C++):

http://msdn.microsoft.com/en-us/library/aa288465(VS.71).aspx

XML Configuration File for .Net Projects

ConfigurationManager class is used to load settings from an XML file at the start of an application. Usually people edit it outside the application (manually with a text editor), however, it is possible to change configuration inside the program:

http://blog.sb2.fr/post/2008/11/29/HowTo-Modify-Configuration-File-Programmatically-with-C.aspx

The Jabber-Net library uses this function to update its configuration file instead:

/// <summary>

/// Write the current connection properties to an XML config file.

/// TODO: Replace this with a better ConfigFile implementation that can write.

/// </summary>

/// <param name=”file”></param>

public void WriteToFile(string file)

{

XmlDocument doc = new XmlDocument();

string name = “JabberClient”;

if (m_xmpp != null)

name = m_xmpp.GetType().Name;

XmlElement root = (XmlElement)doc.CreateElement(name);

doc.AppendChild(root);

WriteElem(root, this);

foreach (DictionaryEntry ent in m_extra)

{

root.AppendChild(doc.CreateElement((string)ent.Key)).InnerText = ent.Value.ToString();

}

XmlTextWriter xw = new XmlTextWriter(file, System.Text.Encoding.UTF8);

xw.Formatting = Formatting.Indented;

doc.WriteContentTo(xw);

xw.Close();

}

More:

More advanced configuration library: http://www.codeproject.com/KB/cs/SysConfiguration.aspx

The Promise of Open Source for our Project

The XMPP protocol has many implementations in many projects, sometimes together with other protocols. Which implementation is the best to build up on.  Among c open source implementations listed, jabberd14 and jabberd2, latter seems to be more recent, more widely adopted, and hence have better community support,  so I played with jabberd2 for a while. Then I also looked at Java implementations. Openfire seems to be more complete than Tigase. It also has a company behind it, rather than a single developer. So currently I am using Openfire.

On the client side development I plan to continue implementing from scratch. Because client program is to be distributed to users and except LGPL opensource, using an open source code would require me to open my source as I had mentioned here. Still, when I am stuck about solving an issue, I can check how they had solved it in one of the open source clients. A well established XMPP client project is: Empathy but it is targeting Linux where as we target customers on Windows initially . LGPL is more common among softare libraries, and there are a few XMPP libraries with LGPL, I initially considered the only C++ one, QXMPP, but now I am using jabber-net and ported my C++/CLI client code to C# as well since utilizing examples for this C# library was easier when I coded in C# as well.

libjingle is a library implementing a useful extention of XMPP protocol, named jingle. Jingle is for initiating and managing peer-to-peer media sessions between two XMPP entities to be used for a wide variety of application types (e.g., voice chat, video chat, file transfer). Luckily, libjingle is made available under a Berkeley-style license, which means you are free to incorporate it into commercial software. Using libjingle, I won’t waste time on the obstacles I would face, like reverse NAT traversal problem I had faced with my own client. As an example, (as described here) libjingle already implements STUN to solve this problem.

Resources:

Comparison of XMPP and other IM Protocols

jabberd2 home page.

Developer documentation for jabberd2.

jabberd2 install guide.

jabberd2 architecture doc. Especially required database tables given at the end.

precompiled windows binaries and suse rpms.

All server implementations list from official website.

Brief info about some Open Source Licences

I was initially thinking I would have to make my program open source if I was using GPL source. GPL requires you to make your source code available only if you are distributing your program. This is a loop-hole benefitting people developing server software for their own use. This loophole is later filled by AGPL, but most software still uses GPL, including XMPP servers I am inquiring about. So I can develop our server on top of open source projects. I would gladly send them patches when I make improvements to their code. I only like not being forced to open all of my source since this might cause security flaws in my code to be detected by hackers.

There is also LGPL, which allows proprietary code to be built upon open code even if you are distributing your program to users. This licence is much less commonly used.

Standard Protocols, Libraries and APIs for Instant Messaging

I will be reading RFCs from IETF standardization committee to learn about the XMPP protocol. However there are preceding protocols to read about. For a few days you will be seeing short summaries from those protocols I will read. Later you may see some info about open source implementations of XMPP.

Twitter explosion a bubble (that will burst)?

Today I received two articles on research about twitter’s success from Karalyn through Bilumni group.

One study found that “more than 60 percent stopped using the free site a month after joining”, and the other article was mentioning exactly my question when I first saw a few status posts on Facebook  “Why would they want to read short messages about what someone ate for breakfast?” and cites uses by marketing people as alternative use for twitter. I have been using twitter for a month now, and 8 out of 10 people who added me to their “follow” list were in fact marketers trying to grab my attention to their tweets.

Another article points to fake celebrities on twitter and tells even god has a twitter account!

I think these are all negative prospects for twitter’s future. I felt the idea is doomed for failure when I had first heard about it considering its usefulness for readers, but push by marketing people may keep it alive. Time will tell…

Notes about C++/CLI syntax

Pointers vs Handles vs Native Types
^ similar to *
% similar to &

N* pn = new N; // allocate on native heap
N& rn = *pn; // bind ordinary reference to native object
R% rr = *hr; // bind tracking reference to gc-lvalueR^ hr = gcnew R; // allocate on CLI heap

Boxing, Unboxing:

Boxing implicit.

Unboxing explicit: need a reinterpret_cast and then dereference.

int i = 1024;
int^ boxedi = i;          //Implicit boxing, boxedi doesn’t point to i;
         // A bit-wise copy is performed          // an int Object is created on the CLR heap         //Can use safe_cast<int^>(i) to explicitly castObject^ o = i; //Another implicit boxing
int k = *safe_cast<int^>(o); //unboxing

//Type-safe: When you box a value-type, // the returned object remembers the original value type.//reinterpret_cast could be used, safe_cast is better// to create safe code (to compile with /clr:safe) Console::WriteLine(o->GetType()); //same with typeid<o>?// Output:
// System.Int32//Can not cast to a different type!

float g = *reinterpret_cast<float^>(o);     //System.InvalidCastException
     // Cap or handle (^) denotes managed object reference     //  (.NET reference types)     //  (instead of *).     // Delete optional,      //  Garbage Collector will handle.     // These don't have a fixed memory address      //  they will be moved around during execution     // Type-safe (you cannot cast a handle to a void^)

Class (and struct) declarations :

class native {}; //Following create CLI types
value class V {};     // “value” => stack types beyond primitive types    // all value types derive from System::ValueType
ref class R {};     // “ref” =>  managed class, instantiated with R^
interface class I {};

native enum & cli class enum:

enum native_Enum { fail, pass };enum class cli_Enum: char { fail, pass}; public enum class cli_SomeColors { Red, Yellow, Blue};

Arrays

array<int>^ ia = gcnew array<int>{1,1,2,3,5,8};     // gcnew retuns handle (new returns pointer)

     // same with cli::array ia = gcnew array<int>(10){2,5};     //creates array of size 10,      //all after first two elements initialized to 0

Interior Pointer

to access to memory within objects.

Lets you do pointer arithmetic.

Dangerous feature.

GC keeps track of changes in memory location and updates the interior pointer unless interior pointer is used by unmanaged code!

const int SIZE = 10;
array<int>^ a = gcnew array<int>(SIZE);
interior_ptr<int> p = &a[0];
for (int i = 0; i < SIZE; ++i, ++p)
   *p = i;

Pin Pointer

Pins the object in memory (GC doesn’t move it around), so the pointer won’t point to invalid memory.

value struct smallInt { int m_ival; ... } si;
pin_ptr<int> ppi = &si.m_ival;   // assign nullptr to pin pointer after its use is over   // so that GC knows it can now move the object around

Iterating Managed Arrays

//For each to iterate, but//Length attribute and the index also available for each (int v in a)
{
  Console::WriteLine("value={0}", v);
}

3D Array Creation

//All cells are initialized to 0 automaticallyarray<int, 3> ^threed = gcnew array<int, 3>(4,5,2);
Console::WriteLine(threed[0,0,0]);

Another example for handles

//This shows a window using WPFApplication().Run(%Window()); // % for referencing 

//===== equivalent toWindow w;
Application().Run(%w);//===== equivalent toWindow ^w = gcnew Window();
Application().Run(w); // no need to reference, pass handle//===== equivalent toWindow ^w = gcnew Window();
Application ^a1 = gcnew Application();
a1->Run(w);//===== equivalent toWindow ^w = gcnew Window();
Application ^a1 = gcnew Application();
a1->Run(w); // similar to pointer handles use ->//===== equivalent toApplication ^a1 = gcnew Application();
Application %a2 = *a1;//like pointers, dereferencing via *
a2.Run(w);

Array of strings and using Tracking reference “^%” to iterate:

similar to “*&” (reference to a pointer) in Standard C++

array<String^>^ arr = gcnew array<String^>(10);
int i = 0;

for each(String^% s in arr)     s = i++.ToString();    //redundant: s = gcnew String( (i++).ToString() );

New parameter for main function

#using <mscorlib.dll>     //you don’t have to include this anymore    //automatically included whenever u compile with /clr
using namespace System;

int main(array<System::String ^> ^args)
{
  System::Console::WriteLine("Hello world");
  return 0;
}

Finalizer vs Deterministic Destructor

Non-managed resources must be Deterministically destroyed.

Deterministic destruction recommended for system-wide limited resources (network/database connections, file streams, etc.).

Even with deterministic destructor also create a finalizer (GC-destructor) that just checks whether the destructor has been invoked and calls it if it hasn’t.

// C++/CLI
ref class MyClass // : IDisposable (added by the compiler)
{
public:
    MyClass();  // constructor
    ~MyClass(); // (deterministic) destructor          // turned into IDisposable.Dispose() by compiler         // corresponds to Dispose() in C#
protected:
    !MyClass(); // finalizer           // corresponds to ~MyClass() in C#
public:
    static void Test()
    {
        MyClass automatic;           //compiler calls constructor here automatically          //will be destroyed by compiler when out of scope

        // Equivalent user code:
        MyClass^ user = gcnew MyClass();
        try { /* Use user here */ }         finally { delete user; }

        // Compiler calls automatic's destructor in the         // finally of a try containing the whole method
    }
};

Restricting what kind of classes can be used for template type(s):

generic<class T> where T : IDisposable
ref class DisposableType
{
   T t;
public:
   DisposableType(T param) {t=param;}
   void Close()
   {
      t->Dispose();      //since there is constraint on what T can be      // no need to cast t to use IDisposable methods
   }
};

Constraints also possible for generic methods:

interface class IPrintable
{
   void Print();
};
ref class Printer
{
public:
   generic<class T> where T : IPrintable
   void PrintDoc(T doc)
   {
      doc->Print(); //no need to cast doc
   }
};

Properties

// LibCLI.h
#pragma once
using namespace System;

namespace LibCLI {
public ref class Person
    {
    private:
       String^ _name;

    public:
        // The constructor.         // there is no _age, only property Age!         // see last line
        Person( String^ name, int age )
        {
            Name  = name;
            Age   = age;
        }
        // property for the member _name
        property String^ Name
        {
            String^ get()
            {
                return _name;
            }

            void set(String^ name)
            {
                _name = name;
            }
        }

        // A trivial property. No need to define
        // the getter and setter, compiler does it for you.
        property int Age;
    };
}

Operator overloading

static for handles; casting to Object^ will remove the overloading semantics

{   String ^s1 = "abc";
   String ^s2 = "ab" + "c";
   Object ^o1 = s1;
   Object ^o2 = s2;
   s1 == s2; // true
   o1 == o2; // false}

Parameter Array

void avg(String ^msg, ... array<int> ^values)
{ // “...” denotes parameter array,   // used to be [ParamArray]attribute before
  int tot = 0;
  for each (int v in values)
    tot += v;
  Console::WriteLine("{0} {1}", msg, tot / values->Length);
}

int main(array<String ^> ^args)
{
  avg("The avg is:", 1,2,3,4,5);
  return 0;
}

Wrapping your C++ classes:

provides deterministic finalization of garbage- collected types. Ex: Wrapping class named Native

ref class Wrapper {
    Native *pn;
public:
    // resource acquisition is initialization
    Wrapper( int val ) { pn = new Native( val ); } 

    // this will do our disposition of the native memory
    ~Wrapper(){ delete pn; }

    void mfunc();
protected:

    // an explicit Finalize() method—as a failsafe
    !Wrapper() { delete pn; }
};

void f1()
{
   // normal treatment of a reference type
   Wrapper^ w1 = gcnew Wrapper( 1024 );

   // mapping a reference type to a lifetime
   Wrapper w2( 2048 ); // no ^ token !

   // just illustrating a semantic difference
   w1->mfunc();
   w2.mfunc();

   // w2 is disposed of here automatically
}

//
// ... later, w1 may be finalized at some point

Another more complete wrapper template ref class with operator overloading, copy constructor and assignment operator

template <typename T>
 public ref class ptr_wrapper sealed
 {
 private:
 	T *m_ptr;
 	ptr_wrapper(T *i_ptr)
 	:m_ptr(i_ptr)
 	{
 		if (i_ptr == 0)
 		{
			throw gcnew System::Exception(				"Trying to initialize				ptr_wrapper with null pointer");
 		}
 	}
 public:
 	ptr_wrapper(const T &i_ref)
 		:m_ptr(new T(i_ref))
 	{
 	}
 	ptr_wrapper(const ptr_wrapper %i_other)
 		:m_ptr(new T(const_cast<const T&>(*i_other)))
 	{
 	}
 	static ptr_wrapper take(T *i_ptr)
 	{
 		return ptr_wrapper(i_ptr);
 	}
 	~ptr_wrapper()
 	{
 		delete m_ptr;
 	}
 	ptr_wrapper % operator = (const ptr_wrapper %other)
 	{
 		if (other.m_ptr != m_ptr)
 		{
 			T* new_ptr = new T(*other);
 			delete m_ptr;
 			m_ptr = new_ptr;
 		}
 	}
 	static T& operator * (ptr_wrapper<T> %inst)
 	{
 		return *(inst.m_ptr);
 	}
 	static const T& operator * (const ptr_wrapper %inst)
 	{
 		return *(inst.m_ptr);
 	}
 	static T* operator -> (ptr_wrapper %inst)
 	{
 		return inst.m_ptr;
 	}
 	static const T* operator -> (const ptr_wrapper<T> %inst)
 	{
 		return inst.m_ptr;
 	}
 };

Command line building

# run the script to set up environment

# use with quotes!

# path assumes VS 2008 (i.e 9.0) installed

”C:\Program Files\Microsoft Visual Studio 9.0\Common7\Tools\vsvars32.bat”

# then compile

cl your_file.cpp /clr

Glossary

Garbage Collector (GC):  reclaims garbage, or memory used by objects that will never be accessed or mutated again by the application.

Common Language Infrastructure (CLI): It is an open specification that defines a runtime environment that allows multiple high-level languages to be used on different computer platforms without being rewritten for specific architectures. [Specification]

Common Type System (CTS): a standard that specifies how Type definitions and specific values of Types are represented in computer memory, so programs in different programming languages can easily share information.

Base Class Library (BCL): a standard library available to all languages using the .NET Framework, comparable in scope to the standard libraries of Java.

Framework Class Library (FCL): a collection of thousands of reusable classes, interfaces and value types, within hundreds of namespaces. BCL is a part of FCL and provide the most fundamental functionality. [Namespaces Poster]

[Class Libraries for .Net 3.5]

Windows Presentation Foundation (WPF):graphical subsystem in .NET Framework 3.0, which uses XAML markup language, for rich user interface development. Microsoft Expression Blend is a designer tool that generates XAML that may be exported into other tools.

[MSDN section]

managed classes: objects are under .NET automatic garbage collection (vs objects the programmer must remember to explicitly destroy)

value type (vs reference type): a data type that directly contain their data (rather than through a reference or pointer), allocated on the stack or allocated inline in a structure.

reference type (vs value type): data type that can be instantiated via dynamic memory allocation.

References

Best Quick Articles are from The Code Project: Quick C++/CLI – Learn C++/CLI in less than 10 minutes and A first look at C++/CLI

Article by Stanley B. Lippman from Microsoft: Pure C++ Hello, C++/CLI

Article by Richard Grimes from DDJ: New Syntax C++ in .NET Version 2

Tutorials on functionx.com [C++/CLI, VC/CLI, also VC.NET]

Resources

MSDN website is full of useful materials, but hard to find, here is some gems I found.

If you would like to teach C++:

Free Ebook by Herbert Schildt,

and other higher level C++ educational materials at MSDN Developer Learning Center: beginner, all levels, all levels

Root for all educational materials: Developer Centers

MSDN Library download
(Full Library of 2GB is on second link)

MUI Information:

MS Technology to support easy translation of developed applications