File system, storage, network, and parsing

Resources packaged within the "JAR" don’t belong in this section of the developer guide, but they’re often confused with Storage and FileSystemStorage, so this is a good place to clarify them.

You can place arbitrary files within the src directory of a Codename One project. The file is packaged into the final distribution. In standard Java SE you can do something like:

This isn’t guaranteed to work on all platforms and will probably fail on some. Instead, use something like:

This isn’t the limitation. You can’t use hierarchies, so something like this fails:

You can’t use relative paths either, so this also fails (notice the missing first slash):

Those limitations exist for portability. iOS and Android handle resources differently, so supporting the full Java SE semantics is unrealistic.

Just like in Java SE, the entries within the "JAR" are read and can’t be modified. You can access the stream, but not the actual file.

Codename One gives you four places to keep data, and picking the wrong one is the usual source of trouble later:

Storage is accessed through the Storage class. It’s a flat, filesystem-like interface that can list, delete, and write named storage entries.

The Storage API also provides convenient methods to write objects to Storage and read them back with readObject and writeObject.

The sample code below demonstrates listing storage content, adding and viewing entries, and deleting entries:

Storage also offers a simple API in the form of the Preferences class. The Preferences class lets developers store simple variables, strings, numbers, booleans, and similar values without writing storage code. This is a common use case in applications. For example, you might need to store a server token:

You can then read the token like this:

The backing store filename defaults to "codenameone.properties", but you can override it by calling Preferences.setPreferencesLocation(String) before your app touches the API. This is useful when you need separate preference namespaces per user or when you want to plug in an encrypted storage layer.

When you have a batch of updates, prefer the Preferences.set(Map<String,Object>) overload so that all values are persisted with a single disk write.

Preferences can also tell interested code when a key changes. Register a PreferenceListener with addPreferenceListener() to react to updates made in other parts of your code or triggered remotely through Codename One Push. Remember to remove listeners you no longer need to avoid leaks.

FileSystemStorage provides file system access. It maps to the underlying OS’s file system API providing most of the common operations expected from a file API somewhat in the vain of java.io.File & java.io.FileInputStream for example: opening, renaming, deleting etc.

Notice that the file system API is somewhat platform specific in its behavior. All paths used the API should be absolute otherwise they aren’t guaranteed to work.

The main reason java.io.File & java.io.FileInputStream weren’t supported directly has a lot to do with the richness of those two APIs. They effectively allow saving a file anywhere, but mobile devices are far more restrictive and don’t allow apps to see/change files that are owned by other apps.

All paths in FileSystemStorage are absolute, this simplifies the issue of portability since the concept of relativity and current working directory aren’t portable.

All URLs use the / as their path separator you try to enforce this behavior even on Windows.

Directories end with the / character and thus can be distinguished by their name.

The FileSystemStorage API provides a getRoots() call to list the root directories of the file system (you can then "dig in" through the listFiles API). For example, this is confusing and unintuitive for developers.

To simplify the process of creating/reading files Codename One provides the getAppHomePath() method. This method allows you to get the path to a directory where files can be stored/read.

You can use this directory to place an image to share as you did in the share sample.

Use Display.openFileChooser() or the shorter CN.openFileChooser() when the user needs to select a general document such as a PDF, text file, private key, or import file. This is different from Display.openGallery(), which is intended for photo and video library media.

The callback source is a String path that can be read with FileSystemStorage.openInputStream(). If the user cancels the picker the event source is null.

The second argument is a comma-separated accept list. It can contain filename extensions such as pdf,txt,p8, MIME types such as application/pdf, or null// for any file. JavaSE uses the native AWT file dialog. Android uses the system document picker and copies content URIs into an app-readable file path when needed. Ports without a native document picker fall back to a Codename One file tree.

No photo-library or media permissions are added by this API. On Android the system document picker grants read access to the selected document. On iOS this API should be implemented with a document picker, not the photo library; ios.NSPhotoLibraryUsageDescription is only needed for openGallery().

A more advanced usage of the FileSystemStorage API can be a FileSystemStorage Tree:

The question of storage vs. file system is often confusing for novice mobile developers. This embeds two separate questions:

The main reasons for the 2 APIs are technical. Many OSes provide 2 ways of accessing data specific to the app and this is reflected within the API. For example: on Android the FileSystemStorage maps to APIs such as java.io.FileInputStream whereas the Storage maps to Context.openFileInput().

The secondary reason for the two APIs is conceptual. FileSystemStorage is more powerful and in a sense provides more ways to fail, this is compounded by the complex on-device behavior of the API. Storage is designed to be friendlier to the uninitiated and more portable.

You should pick Storage unless you have a specific need that prevents it. Some APIs such as Capture expect a FileSystemStorage URI so in those cases this would also be a need.

Another case where FileSystemStorage is beneficial is the case of hierarchy or native API usage. If you need a directory structure or need to communicate with a native API the FileSystemStorage approach is easier.

To summarize the differences between the 3 file storage options:

SQLite is a small SQL database designed for embedding into an application. Codename One exposes it through com.codename1.db, an API similar in spirit to JDBC but much smaller, since the abstractions JDBC needs for pluggable drivers make no sense for a local file.

Use it when you have tabular data, need to query it, or have enough of it that loading the lot into memory is unreasonable. For a handful of values Storage is simpler.

The Database API opens a database by name:

Some SQLite apps ship with a "ready made" database. You allow you to replace the DB file by using the code:

You can then use the FileSystemStorage class to write the content of your DB file into the path. Notice that it must be a valid SQLite file!

This is useful for applications that need to synchronize with a central server or applications that ship with a large database as part of their core product.

Working with a database is pretty trivial, the application logic below can send arbitrary queries to the database and present the results in a Table. You can probably integrate this code into your app as a debugging tool:

Pass a DatabaseConfig to encrypt a database at rest. Check that the platform can do it first:

There are three ways to key a database.

Encryption protects data at rest, and nothing else. It’s no defense against a rooted or jailbroken device, a debugger attached to the running process, or a memory dump, because the key is in memory for as long as the database is open.

Managed keys aren’t recoverable. Losing the key store entry, through an application uninstall, a device wipe or, on Android, restoring a backup onto a different device, leaves the database permanently unreadable. Android keystore keys can’t be exported; iOS keychain entries do survive an encrypted backup and restore. If the data has to outlive the device, use a passphrase your user or your server holds.

Nothing is added to an application that doesn’t use it. The build scans your own classes — not the framework’s — and decides from what they reference:

An application that already ships a plaintext database can convert it in place:

Database.decrypt is the reverse, and Database.changeKey re-keys a database that’s already open. An interruption leaves the original file intact, and schema metadata such as PRAGMA user_version survives.

How that’s achieved differs by platform, which matters only if you’re watching the filesystem. Most platforms re-key the file in place. Android can’t: its SQLCipher engine re-keys only between two encrypted states and refuses to convert to or from plaintext, so the port writes the converted database alongside the original and swaps it in once it’s complete. A .cn1migrate file next to your database means a conversion was interrupted partway; the original is still the live one, and the leftover is cleaned up on the next attempt.

Every platform reads and writes one format, so a database created on a phone can be pulled off the device and opened in the simulator while you are debugging. The parameters are fixed: AES-256 in CBC mode, PBKDF2-HMAC-SHA512 key derivation at 256000 iterations, a 4096 byte page size and per-page HMAC-SHA512. This is the SQLCipher 4 format, so standard tools such as the sqlcipher command line client can open these files too.

Transactions are flat, because that’s the only model all the underlying engines can express. A beginTransaction() while one is already open throws, and so does a commitTransaction() or rollbackTransaction() with none open. Both end the transaction and return the connection to autocommit, and closing a database with one open rolls it back.

A commit can fail. SQLite checks a deferred constraint when the transaction commits, not when the offending row is inserted, so that’s where the failure surfaces. When it does, the transaction is over: the port ends it for you and beginTransaction() works again straight away. Don’t roll back afterward, because there’s nothing left to roll back and the call throws. The engines don’t agree here on their own — Android ends the transaction before it reports the failure, while the SQLite C API and JDBC leave it open — so the port reconciles them and your code sees one behaviour.

ATTACH makes another database part of the connection, and SQLite holds it until you detach it or close the connection. Codename One counts an attached database as open, so deleting it, converting it or changing its key is refused while it’s attached, the same as for the database you opened directly. That’s the protection you want: unlinking a file SQLite still has open loses everything written through it.

SQLite lets you attach one file under several schema names at once. The file stays protected until the last of them is detached, so a DETACH of one name doesn’t release a database the connection still holds under another.

Write the filename as a single quoted literal, or bind it as a parameter. SQLite also accepts an expression there — ATTACH '/data/' || name AS aux — but nothing can tell which file that names until the engine evaluates it, which is after the database is already attached and too late to refuse. Those are rejected, and the message says so; build the name in your own code and pass it as a parameter.

Run ATTACH and DETACH through execute(). executeQuery() refuses them, because a cursor runs its statement when it’s stepped — so the attachment would happen after the bookkeeping, and the file could then be deleted while SQLite still holds it.

A path that runs through a symbolic link identifies the same database as the resolved path does. The engine resolves links when it reports what it opened, so Codename One resolves them too before it decides two names are one file — on the platforms where the engine does. Windows is the exception, because SQLite’s Windows layer collapses . and .. as text and steps over a junction rather than through it, so the unresolved name is what identifies the file there.

Name the file with an absolute path from Database.getDatabasePath(). A relative name is resolved by SQLite against the process working directory, which is a different place on every platform and isn’t something a Codename One application controls, while Codename One would look for it in the database directory — so the database protected wouldn’t be the database attached. Those are rejected on every port whose engine has a working directory to resolve against; in the browser, where a name is an entry in a storage pool and means the same thing either way, they’re accepted. If an attachment turns out to name a database that something else is deleting or converting, it’s detached again and the next call on that connection throws to tell you.

A Database and its cursors aren’t thread safe. Either give each thread its own connection, or wrap one in ThreadSafeDatabase, which routes every call through a single worker thread:

The wrapper costs a thread handoff per call, so a connection per thread is faster when the threads don’t genuinely need to share one.

Text is stored and read back exactly as written, for every character a Java String can hold: accented characters, scripts outside ASCII, emoji, and even the character with code point zero, which SQLite stores in a TEXT value like any other. A string written on one platform reads back identical on every other.

Reading a column the result set doesn’t have — a negative index, or one past the last column — raises an IOException. It’s never reported as a null value, because SQLite’s own C interface answers an out-of-range index with SQL NULL, and passing that on would make a mistyped index impossible to tell apart from stored data.

Iterating forward with next() costs the same everywhere. Seeking backwards or to an absolute row is cheap on Android, which keeps a window of rows, and costs time proportional to the distance from the start elsewhere, because a SQLite statement only steps forward and seeking back means rewinding and stepping again. Prefer next() for large result sets.

One consequence is worth knowing: because a backward seek re-runs the statement, a cursor is a repeatable read only inside a transaction. Outside one, another thread can write between the two passes, changing what the second pass sees.

There’s a limit on Android that follows from the same mechanism. A statement that changes the database and returns rows — INSERT …​ RETURNING and its relatives — runs once, and Android reads its rows through a window it can only refill by running the statement again. The rows that fit in the first window are therefore readable and the rest aren’t: reaching them would apply the insert a second time. Codename One refuses that rather than doing it, and says so in the error, including that the changes are already applied and the statement must not be retried. If the result set may be large, run the statement with execute() and query for what you need afterward. Every other port steps through such a statement once and has no such limit.

This API predates the portable contract, and its behaviour used to differ between platforms. If your application depends on the old behaviour, set the db.legacy build hint, or call Database.setLegacyBehavior(true) before opening any database. It restores each platform’s previous behaviour exactly.

An Ant project that uses com.codename1.db is built in compatibility mode automatically. Ant projects predate the portable contract, so a rebuild shouldn’t change how their queries behave unless you ask for it. A Maven project gets the portable contract, which is the documented default everywhere else in this section.

The build server decides this from the project itself, and says so in the build log when it turns compatibility mode on. It only applies when your application actually references com.codename1.db; an Ant project with no database is unaffected.

The build hint always wins, in either direction:

An Ant project can therefore opt in to the portable contract by setting db.legacy=false, and a Maven project can pin compatibility mode with db.legacy=true. Setting it explicitly is worth doing once you’ve decided, because it makes the choice visible in the project rather than inferred from how it’s built. If you migrate an Ant project to Maven without setting the hint, the default flips, so set it before you migrate if you want the behaviour to stay put.

The hint works in the simulator too, so you can test either mode without a device.

The flag doesn’t cover defects, nor capabilities that used to throw and now work: last(), prev() and position(int) on iOS and the simulator, reading blobs on iOS, blob query parameters, binding null inside a String[], the existence of a database at all on the native Windows and Linux ports, or seeking backwards through a statement that writes, which used to run that statement a second time and repeat what it wrote. No application can depend on those.

Everything above is behaviour Codename One implements, so it can be made the same everywhere. Two things are decided when the SQLite engine itself is compiled, and Codename One only compiles one of those engines — Android uses the platform’s, the simulator uses the JDBC driver’s, and iOS uses Apple’s unless the application encrypts:

Both are easy to settle for yourself: SELECT "value" either returns the string or reports an error, and PRAGMA foreign_keys reports whether they’re on. Codename One’s own port conformance run reports both for every platform it covers.

One of the more common problems in Network programming is spawning a new thread to handle the network operations. In Codename One this is done seamlessly and becomes unessential thanks to the NetworkManager.

NetworkManager effectively alleviates the need for managing network threads by managing the complexity of network threading. The connection request class can be used to ease web service requests when coupled with the JSON/XML parsing capabilities.

That convenience comes with one rule worth learning before the API: the callbacks that read and write the request run on a network thread, while postResponse, a response listener and an error listener all run on the EDT.

To open a connection one needs to use a ConnectionRequest object, which has some similarities to the networking mechanism in JavaScript but is obviously somewhat more elaborate.

You can send a get request to a URL using something like:

Notice that you can also implement the same thing and much more by avoiding the response listener code and instead overriding the methods of the ConnectionRequest class which offers many points to override for example:

NetworkManager also supports synchronous requests which work in a similar way to Dialog through the invokeAndBlock call and thus don’t block the EDT ^[8] illegally. For example: you can do something like this:

Notice that in this case the addToQueueAndWait method returned after the connection completed. Also notice that this was totally legal to do on the EDT!

Each request can override the global timeout using setTimeout(int), which expects a duration in milliseconds. This is useful when you’re talking to endpoints that occasionally need a longer window than the default NetworkManager setting. You can also configure how many times Codename One should retry a failing call by using setSilentRetryCount(int). When the silent retry limit is reached the standard error handling kicks in (for example: listeners fire, fail dialogs show, etc.).

By default the NetworkManager launches with a single network thread. This is enough for simple applications that don’t do too much networking but if you need to fetch many images concurrently and perform web services in parallel this might be an issue.

To update the number of threads use:

The callbacks you override on a ConnectionRequest occur on the network thread and not on the EDT!

The callbacks you register are the other way round. postResponse(), designed to update the UI after the networking code completes, is dispatched back to the EDT with callSerially, and so are the listeners added through addResponseListener and NetworkManager.addErrorListener.

NetworkManager includes two APIs for querying VPN state:

Both Android and iOS implementations rely on platform networking metadata and interface-name patterns (for example: tun, ppp, utun, ipsec). This means the result can be wrong in either direction:

Use this API for UX or diagnostics (for example: warnings, telemetry, or logging), not for access control or fraud prevention by itself.

HTTP/S is a complex protocol that expects complex encoded data for its requests. Codename One tries to simplify and abstract most of these complexities behind common sense APIs while still providing the full low-level access you would expect from such an API.

HTTP supports many "request methods," most commonly GET & POST but also a few others such as HEAD, PUT, DELETE etc.

Arguments in HTTP are passed differently between GET and POST methods. That’s what the setPost method in Codename One determines, whether arguments added to the request should be placed using the GET semantics or the POST semantics.

If you continue your example from above you can do something like this:

This will implicitly add a get argument with the content of value. Notice that you don’t care what value is. It’s implicitly HTTP encoded based on the get/post semantics. In this case it will use the get encoding since you passed false to the constructor.

A simpler implementation could do something like this:

This would be almost identical but doesn’t provide the convenience for switching back and forth between GET/POST and it isn’t as fluent.

You can skip the encoding in complex cases where server code expects illegal HTTP characters (this happens) using the addArgumentNoEncoding method. You can also add many arguments with the same key using addArgumentArray.

As you explained above, the setPost() method allows you to manipulate the get/post semantics of a request. This implicitly changes the POST or GET method submitted to the server.

For example, if you wish to have finer grained control over the submission process for example: for making a HEAD request you can do this with code like:

When communicating with HTTP servers you often pass data within headers for authentication/authorization but also to convey various properties.

Some headers are built-in as direct APIs for example: content type is directly exposed within the API since it’s a pretty common use case. You can set the content kind of post request using:

You can also add any arbitrary header type you want, for example: a common use case is basic authorization where the authorization header includes the Base64 encoded user/password combination as such:

This can be tedious to do if you want all requests from your app to use this header. For this use case you can use:

Server returned headers are a bit trickier to read. You need to subclass the connection request and override the readHeaders method for example:

Here you can extract the headers one by one to handle complex headers such as cookies, authentication etc.

As you noticed above practically all the methods in the ConnectionRequest throw IOException. This allows you to avoid the try/catch semantics and let the error propagate up the chain so it can be handled uniformly by the application.

There are two distinct places where you can handle a networking error:

Notice that the NetworkManager error handler takes precedence thus allowing you to define a global policy for network error handling by consuming errors.

For example: to block all network errors from showing anything to the user you could do something like this:

The error listener is invoked first with the NetworkEvent matching the error. Consuming the event prevents it from propagating further down the chain into the ConnectionRequest callbacks.

You can also override the error callbacks of the various types in the request for example: for a server error code you can do:

Another approach is to use the setFailSilently(true) method on the ConnectionRequest. This will prevent the ConnectionRequest from displaying any errors to the user. It’s a powerful strategy if you use the synchronous version of the APIs for example:

When you get an error code that isn’t 200/300 you ignore the result. This is problematic as the result might contain information you need. For example: many webservices provide further XML/JSON based details describing the reason for the error code.

Calling setReadResponseForErrors(true) will trigger a mode where even errors will receive the readResponse callback with the error stream. This also means that APIs like getData and the listener APIs will also work in case of error.

Gzip is a common compression format based on the lz algorithm, it’s used by web servers around the world to compress data.

Codename One supports GZIPInputStream and GZIPOutputStream, which allow you to compress data seamlessly into a stream and extract compressed data from a stream. This is useful and can be applied to every arbitrary stream.

Codename One also features a GZConnectionRequest, which will automatically unzip an HTTP response if it’s indeed gzipped. Notice that some devices (iOS) always request gzip’ed data and always decompress it for you, but for iOS it doesn’t remove the gziped header. The GZConnectionRequest is aware of such behaviors so it’s better to use that when connecting to the network (if applicable).

By default GZConnectionRequest doesn’t request gzipped data ( unzips it when its received) but its pretty easy to do so add the HTTP header Accept-Encoding: gzip for example:

Do the rest as usual and you should have smaller responses from the servers.

MultipartRequest tries to simplify the process of uploading a file from the local device to a remote server.

You can always submit data in the buildRequestBody but this is flaky and has some limitations in devices/size allowed. HTTP standardized file upload capabilities through the multipart request protocol, this is implemented by countless servers and is well documented. Codename One supports this out of the box:

Since you assume most developers reading this will be familiar with Java here is the way to implement the multipart upload in the servlet API:

This is server-side Java rather than Codename One, so it isn’t one of the compiled examples:

Codename One has many built in parsers for JSON, XML, CSV & Properties formats. You can use those parsers to read data from the Internet or data that’s shipping with your product. For example: use the CSV data to setup default values for your application.

All your parsers are designed with simplicity and small distribution size; they don’t check and will fail in odd ways when faced with broken data. The main logic behind this is that validation takes up CPU time on the device where CPU is a precious resource.

CSV is probably the easiest to use, the "Comma Separated Values" format is a list of values separated by commas (or some other character) with new lines to show another row in the table. These map well to an Excel spreadsheet or database table and are supported by default in all spreadsheets.

To parse a CSV use the CSVParser class as such:

The data contains a two-dimensional array of the CSV content. You can change the delimiter character by using the CSVParser constructor that accepts a character.

The JSON ("JavaScript Object Notation") format is popular on the web for passing values to/from webservices since it works so well with JavaScript. Parsing JSON is as easy but has two different variations. You can use the JSONParser class to build a tree of the JSON data as such:

The response is a Map containing a nested hierarchy of Collection (java.util.List), Strings and numbers to represent the content of the submitted JSON. To extract the data from a specific path iterate the Map keys and recurs into it.

The sample below uses results from an API of ice and fire that queries structured data about the "Song Of Ice & Fire" book series. Here is a sample result returned from the API for the query https://www.anapioficeandfire.com/api/characters?page=5&pageSize=3 :

You will place that into a file named "anapioficeandfire.json" in the src directory to make the next sample simpler:

An alternative approach is to use the static data parse() method of the JSONParser class and implement a callback parser for example:

JSONParser.parse(reader, callback);

Notice that a static version of the method is used! The callback object is an instance of the JSONParseCallback interface, which includes many methods. These methods are invoked by the parser to show internal parser states, this is like the way traditional XML SAX event parsers work.

For typed DTO serialization use the @Mapped annotation framework (see JSON / XML Object Mapping) plus Mappers.toJson(…​). For ad-hoc maps / lists where a build-time Mapper would be overkill, the com.codename1.io.JSONWriter class is the complement of JSONParser:

For tiny request bodies the fluent builders are usually faster to read than a Map literal:

Strings are double-quoted with the standard backslash escapes for ", \, \n, \r, \t, \b, \f, and control chars below 0x20 are emitted as \u00xx. The writer doesn’t pretty-print; if you need indented output, run the result through an external formatter at debug time.

The XMLParser started its life as an HTML parser built for displaying mobile HTML. That usage has since been deprecated but the parser can still parse many HTML pages and is "loose" in verification. This is both good and bad as the parser will work with invalid data without complaining.

The simplest usage of XMLParser looks a bit like this:

The Element contains children and attributes. It represents a tag within the XML document and even the root document itself. You can iterate over the XML tree to extract the data from within the XML file.

You’ve had a great sample of working with XMLParser in the Tree Section of this guide.

XMLParser has the complimentary XMLWriter class which can generate XML from the Element hierarchy. This allows developers to mutate (change) the elements and save them to a writer stream.

The Result class provides a subset of XPath, but it isn’t limited to XML documents, it can also work with JSON documents, and even with raw Map objects.

Start by demonstrating how to process a response from the Google Reverse Geocoder API. Start with this XML snippet:

You want to extract some data above into simpler string results. You can do this using:

If you’re at all familiar with processing responses from webservices, you will notice that what would require many lines of code of selecting and testing nodes in regular java can now be done in a single line using the new path expressions.

In the code above, input can be any of:

To use the expression processor when calling a webservice, you could use something like the following to parse JSON (notice this is interchangeable between JSON and XML):

The returned JSON looks something like this (notice it’s snipped because the data is too long):

The XML processor handles global selections by using a double slash anywhere within the expression, for example:

It’s also possible to do some more complex expressions. You’ll use the following XML fragment for the next batch of examples:

Above, if you want to select the IDs of all players that are ranked in the top 2, you can use an expression like:

If a document is ordered, you might want to select nodes by their position, for example:

It’s also possible to select parent nodes, by using the .. expression. For example:

Above, you globally find a lastname element with a value of ‘Hewitt’, then grab the parent node of lastname which happens to be the player node, then grab the ID attribute from the player node. Or, you could get the same result from the following simpler statement:

A predicate can’t hold another predicate. An expression such as //player[//address[country/isocode='CA']]/@id is rejected with Syntax error: unclosed predicate, because the outer predicate is split at its comparator before it’s tokenized and that cuts the inner one in half. Select the inner nodes first and match on what comes back.

To select a node based on the existence of an attribute:

Above, you selected the IDs of all ranked players. Conversely, you can select the non-ranked players like this:

There’s no matching test for a child node. A predicate that names one with no comparison in it — //player[middlename] — is rejected with Could not create a comparator, so test for a child by comparing it to something, or select the child and work from what comes back.

Keep in mind that the Codename One path expression language isn’t a full implementation of XPath 1.0, but does already handle many of the most useful features of the specification.

Properties files are standard key/value pairs encoded into a text file. This file format is familiar to Java developers and the Codename One specific version tries to be as close as possible to the original Java implementation.

Notice that properties file both in Java proper and in Codename One don’t support non-ascii characters. To encode Unicode values into the properties file format you should use the native2ascii tool that ships with the JDK.

One major difference between standard Java properties and the ones in Codename One is that Codename One sorts properties alphabetically when saving. Java uses random order based on the Hashtable natural ordering.

This was done to provide consistency for saved files.

Codename One includes a Network Monitor tool which you can access through the simulator menu option. This tool reflects all the requests made through the connection requests and displays them in the left pane. This allows you to track issues in your code/web service and see everything the is "going through the wire."

This is a useful tool for optimizing and for figuring out what’s happening with your server connection logic.

A common task is file download to storage or filesystem.

The Util class has simple utility methods:

downloadUrlToFileSystemInBackground, downloadUrlToStorageInBackground, downloadUrlToFile & downloadUrlToStorage.

These all delegate to a feature in ConnectionRequest:

ConnectionRequest.setDestinationStorage(fileName) & ConnectionRequest.setDestinationFile(fileName);

Both of which simplify the whole process of downloading a file.

Codename One has many ways to download an image and the general recommendation is the URLImage. For example, the URLImage assumes that you know the size of the image in advance or that you’re willing to resize it. In that regard it works great for some use cases but not so much for others.

The download methods mentioned above are great alternatives but they’re a bit verbose when working with images and don’t provide fine grained control over the ConnectionRequest for example: making a POST request to get an image.

To make this process simpler there is a set of helper methods in ConnectionRequest that downloads images directly.

These methods complement the Util methods but go a bit further and feature terse syntax for example: you can download a ConnectionRequest to Storage using code like this:

This effectively maps the ConnectionRequest directly to a SuccessCallback for further processing.

URLImage is great. It changed the way you do some things in Codename One.

For example, when it was introduced there was no support for the cache filesystem or for the JavaScript port. The cache filesystem is probably the best place for images of URLImage so supporting that as a target is an obvious choice, but JavaScript seems to work so why would it need a special case?

JavaScript already knows how to download and cache images from the web. URLImage is actually a step back from the things a good browser can do so why not use the native abilities of the browser when you’re running there and fallback to using the cache filesystem if it’s available and as a last resort go to storage…​

That’s what the new method of URLImage does:

A few important things you need to notice about this method:

If you do use this approach it would be far more efficient when running in the JavaScript port and will make better use of caching in most OSes.

The Rest API makes it easy to invoke a RESTful webservice without many of the nuances of ConnectionRequest. You can use it to define the HTTP method and start building based on that. To get a parsed JSON result from a URL you could do:

For a lot of REST requests this will fail because you need to add an HTTP header indicating that you accept JSON results. You have a special case support for that:

POST requests use the same builder pattern:

Notice the usage of post and the body builder method. MANY methods exist in the builder class that cover pretty much everything you would expect and then some when it comes to the needs of rest services.

Some highlights that are easy to miss:

When the endpoint returns a top-level JSON array (for example [{...}, {...}]), use fetchAsJsonList rather than fetchAsJsonMap. The underlying JSONParser wraps top-level arrays under a synthetic "root" key; fetchAsJsonList unwraps that envelope for you so the callback receives the array directly:

There is no separate builder for the ambiguous "object or array" case: when the response shape isn’t known up-front, use fetchAsJsonMap and branch on data.get("root") instanceof List.

Rest.fetchAsJsonMap returns a generic Map<String, Object> that the caller casts and inspects key by key. Once your DTOs are @Mapped-annotated (see JSON / XML Object Mapping), fetchAsMapped returns the typed object directly:

For endpoints that return a list of DTOs, use fetchAsMappedList:

Both builders fall back to the untyped fetchAsJsonMap / fetchAsJsonList path internally and then route every map element through the build-time mapper that the @Mapped annotation processor generated. If a mapper isn’t registered for the requested class (typical cause: the process-annotations Mojo didn’t run on the class, or the class isn’t @Mapped-annotated), the callback completes with null data — so always inspect response.getResponseCode() rather than response.getResponseData() to differentiate a server error response from the missing-mapper case.

The code in the kitchen sink webservice sample was updated to use this API. The result is shorter and more readable without sacrificing anything.

The best way to explain the usage of this API is through a concrete "real world" example. Twilio provides many great telephony-oriented webservices to developers. One of those is an SMS sending webservice which can be useful for things such as "device activation."

To get started you would need to sign up to Twilio and have the following 3 variable values:

You can now send hello world as an SMS to the end user. Once this is in place sending an SMS through REST is a matter of using the Rest API:

Notice that this is equivalent of this "curl" command:

That’s pretty cool as the curl command maps almost directly to the Rest API call!

What you do here is actually pretty trivial, you open a connection the api messages URL. You add arguments to the body of the post request and define the basic authentication data.

The result is in JSON form you ignore it since it isn’t that important but it might be useful for error handling. This is a sample response (redacted keys):

Notice the error message entry which is null meaning there was no error, if there was an error you’d have a message there or an error code that isn’t in the 200-210 range.

This should display an error message to the user if there was a problem sending the SMS:

The protocol it implements is a compact binary one. Method arguments were restricted to a small set of types, extended through object externalization, and every generated method came in two forms: one that blocks and throws IOException, and one that takes a callback and reports through it.

Caching server data locally is a huge part of the advantage a native app has over a web app. This is non-trivial as it requires a delicate balance if you want to test the server resource for changes.

HTTP provides two ways to do that the ETag and Last-Modified. While both are great they’re non-trivial to use and by no definition seamless.

Connection request has an experimental feature that allows you to set the caching mode to one of 4 states either globally or per connection request:

You can toggle these in the specific request by using setCacheMode(CachingMode) and set the global default using setDefaultCacheMode(CachingMode).

There are many methods of interest to keep an eye for:

This returns the cached data. This is invoked to implement readResponse(InputStream) when running offline or when you detect that the local cache isn’t stale.

The smart mode implements this and will fetch the right data. For example, the manual mode doesn’t store the data and relies on you to do so. In that case you need to return the data you stored at this point and must implement this method for manual mode.

This is a callback that’s invoked to show a cache hit, meaning that you already have the data.

The default implementation still tries to call all the pieces for compatibility (for example: readResponse). For example, if this is unnecessary you can override that method with a custom implementation or even a blank implementation to block such a case.

These methods are pretty self-explanatory. Notice one caveat though…​

When you download a file or a storage element you don’t cache them and rely on the file/storage element to be present and serve as "cache." When purging you won’t delete a file or storage element you downloaded and thus these might remain.

For example, you do remove the ETag and Last-Modified data so the files might get refreshed the next time around.

The CachedDataService allows caching data and updating it if the data changed on the server. The download APIs won’t check for update if there is a local cache of the data (for example: URLImage always uses the local copy). This isn’t a bad thing, it’s pretty efficient.

For example, it might be important to update the image if it changed but you still want caching.

The CachedDataService will fetch data if it isn’t cached locally and cache it. When you "refresh" it will send a special HTTP request that will send back the data if it has been updated since the last refresh:

Codename One provides the Externalizable interface, which is like the Java SE Externalizable interface. This interface allows an object to declare itself as Externalizable for serialization (so an object can be stored in a file/storage or sent over the network). For example, due to the lack of reflection and use of obfuscation these objects must be registered with the Util class.

Codename One doesn’t support the Java SE Serialization API due to the size issues and complexities related to obfuscation.

The major objects that are supported by default in the Codename One Externalizable are: String, Collection, Map, ArrayList, HashMap, Vector, Hashtable, Integer, Double, Float, Byte, Short, Long, Character, Boolean, Object[], byte[], int[], float[], long[], double[].

Externalizing an object such as h below should work fine:

For example, notice that some things aren’t polymorphic for example: if you will externalize a String[] you will get back an Object[] since String arrays aren’t detected by the implementation.

Implementing the Externalizable interface is important when you want to store a proprietary object. In this case you must register the object with the com.codename1.io.Util class so the externalization algorithm will be able to recognize it by name by invoking:

An Externalizable object must have a default public constructor and must implement the following 4 methods:

The getVersion() method returns the current version of the object allowing the stored data to change its structure in the future (the version is then passed when internalizing the object). The object ID is a String uniquely representing the object; it corresponds to the class name (in the example above the Unique Name should be MyClass).

Developers need to write the data of the object in the externalize method using the methods in the data output stream and read the data of the object in the internalize method for example:

Since strings might be null sometimes you also included convenience methods to implement such externalization. This effectively writes a boolean before writing the UTF to show whether the string is null:

Assuming you added a new date field to the object you can do the following. Notice that a Date is a long value in Java that can be null. For completeness the full class is presented below:

Notice that you need to check for compatibility during the reading process as the writing process always writes the latest version of the data.

Codename One provides many tools to simplify the path between networking/IO & GUI. A common task of showing a wait dialog or progress sign while fetching network data can be simplified by using the InfiniteProgress class for example:

The process of showing a progress bar for a long IO operation such as downloading is automatically mapped to the IO stream in Codename One using the SliderBridge class.

The SliderBridge class can bind a ConnectionRequest to a Slider and effectively show the progress of the download. For example:

Codename One includes a Log API that allows developers to invoke Log.p(String) or Log.e(Throwable) to log information to storage.

As part of the premium cloud features it’s possible to invoke Log.sendLog() to email a log directly to the developer account. Codename One can do that seamlessly based on changes printed into the log or based on exceptions that are uncaught or logged for example:

This code will send a log every 2 minutes to your email if anything was changed. You can place it within the init(Object) method of your application.

For a production application you can use Log.REPORTING_PRODUCTION which will email the log on exception.

At this moment Codename One supports TCP sockets. Server socket (listen/accept) is available on Android and the simulator but not on iOS.

You can check if Sockets are supported using the Socket.isSupported(). You can test for server socket support using Socket.isServerSocketSupported().

To use sockets you can use the Socket.connect(String host, int port, SocketConnection eventCallback) method.

To listen on sockets you can use the Socket.listen(int port, Class scClass) method which will instantiate a SocketConnection instance (scClass is expected to be a subclass of SocketConnection) for every incoming connection.

This simple example allows you to create a server and a client assuming the device supports both:

In standard Java you have a POJO (Plain Old Java Object) which has getters/setters for example: you can have a simple Meeting class like this:

That’s a classic POJO and it’s the force that underlies JavaBeans and a few tools in Java.

The properties are effectively the getters/setters for example: subject, when etc. But properties have many features that are crucial:

These features are crucial since properties allow you all kinds of magic for example: hibernate/ORM uses properties to bind Java objects to a database representation, jaxb does it to parse XML directly into Java objects and GUI builders use them to customize UI’s visually.

POJO’s don’t support most of that so pretty much all Java based tools use a lot of reflection & bytecode manipulation. This works but has a lot of downsides for example: say you want to map an object both to the Database and to XML/JSON.

Would the bytecode manipulation collide?

Would it result in duplicate efforts?

And how do you write custom generic code that uses such abilities? Do you need to manipulate the VM?

These are all abstract ideas, lets look at how you think properties should look in Java and how you can gain from this moving forward.

This is the same class as the one above written with properties:

This looks a bit like a handful, so start with usage which might clarify a few things, then dig into the class itself.

With a plain object you would have written this:

With properties you do this:

At first glance it looks like you created public fields (which you did) but if you will look at the declaration you will notice the final keyword:

That final is what makes the field safe to expose: meet.subject = otherValue doesn’t compile, because there’s nothing to assign to.

All setting/getting must happen through the set/get methods and they can be replaced. For example: this is valid syntax that prevents setting the property to null and defaults it to an empty string:

Since Property is a common class it’s pretty easy for introspective code to manipulate properties. For example, it can’t detect properties in an object without reflection.

That’s why you have the index object and the PropertyBusinessObject interface (which defines getPropertyIndex).

The PropertyIndex class provides metadata for the surrounding class including the list of the properties within. It allows enumerating the properties and iterating over them making them accessible to all tools.

Furthermore all properties are observable with the property change listener. You can write this to instantly print out any change made to the property:

That’s the simple stuff that can be done with properties, but they can do much more!

For starters all the common methods of Object can be implemented with almost no code:

This is easy thanks to introspection…​

You already have some simple code that can convert an object to/from JSON Maps for example: this can fill the property values from parsed JSON:

And vice versa:

You also have a simple ORM solution that maps values to table columns and can create tables. It’s no hibernate but sqlite isn’t big iron so it might be good enough.

One of the problematic issues with constructors is that any change starts propagating everywhere. If you have fields in the constructor and you add a new field later you need to keep the old constructor for compatibility.

Codename One added a new syntax:

That’s why every property in the definition needed the Meeting generic and the set method returns the Meeting instance…​

You’re pretty conflicted on this feature and are thinking about removing it.

Without this feature the code would look like this:

Suppose you have an object called Contacts which includes contact information for example:

Standard Java Objects can be serialized in Codename One by implementing the Codename One Externalizable interface. You also need to register the Externalizable object so the implementation will be aware of it. Codename One business objects are seamlessly Externalizable and you need to register them.

For example: you can do something like this in your init(Object) method:

After you do that once you can write/read contacts from storage if you so want:

This will obviously also work for things like List<Contact> etc.

Writing SQL code can be tedious. Which is why SQLMap is such an important API for some of you. SQLMap allows CRUD (Create, Read, Update, Delete) operations on the built-in SQLite database using property objects.

If you continue the example from above to show persistence to the SQL database you can do something like this:

In the above code you do the following:

You can then add entries to the contact table using:

You can update an entry using:

And delete an entry using:

Listing the entries is more interesting:

The arguments for the select method are:

There are many more configurations where you can fine tune how a specific property maps to a column etc.

The SQLMap API is simplistic and doesn’t try to be Hibernate/JPA for mobile. Basic things aren’t available now and won’t work. This isn’t necessarily a problem as mobile databases don’t need to be as powerful as server databases.

There’s no way to map an object to another object in the database with the one-many or one-one relationships JPA offers. The SQLMap API isn’t suited to that level of mapping.

SQLite is sensitive to threading issues on iOS, so SQLMap issues its calls in process rather than moving them off the caller’s thread. That can be a problem for larger data sets, because the calls then run on the EDT.

Altering a table to match a schema that changed after the table was created isn’t supported. Migrating such a schema means issuing that SQL directly.

Functions, joins, transactions and a good deal of other SQL sit outside the mapping.

SQL can be used directly for any of that. A transaction begun before an insert, update or delete works as advertised, but the mapping is unaware of a rollback, so the data has to be re-fetched afterward.

The mapping covers autoincrement and the cases that map cleanly onto a table; a use case that needs more than that subset is better served by dropping to SQL directly.

Nothing is cached, so selecting the same row twice produces two separate instances. Update both and one of them wins, with no indication that the other was discarded.

That means the application has to keep track of the objects it holds, so a modified version isn’t left sitting beside an older one.

Some objects make sense as global objects, you can use the Preferences API to store that data directly but then you don’t have the type safety that property objects bring to the table. That’s where the binding of property objects to preferences makes sense. For example: say you have a global Settings property object you can bind it to preferences using:

If settings has a property called companyName it would bind into Preferences under the Settings.companyName entry.

You can do some more elaborate bindings such as:

This would customize all entry keys to start with MySettings- instead of Settings.. This would also set the company name entry to company so in this case instead of Settings.companyName you’d have MySettings-company.

One of the bigger features of properties are their ability to bind UI to a property. For example: if you continue the sample above with the Contact class, say you have a text field on the form and you want the property (which you mapped to the database) to have the value of the text field. You could do something like this:

That would work but what if you changed the property value, that wouldn’t be reflected in the text field?

Also that works for text field but what about other types for example: numbers, check boxes, pickers etc. This becomes a bit more tedious with those.

Binding makes this all seamless. For example: the code above can be written as:

The cool thing is that this works with many component types and property types almost magically. Binding works by using an adapter class to convert the data to/from the component. The adapter itself works with a generic converter for example: this code:

Seems like the one above but it takes a String that’s returned by the text field and seamlessly converts it to the integer needed by rank. This also works in the other direction…​

You can build a UI that would allow you to edit the Contact property in memory:

You skipped a couple of fact about the bind() method. It has an additional version that accepts a ComponentAdapter which allows you to adapt the binding to any custom 3rd party component. That’s a bit advanced for now but I might discuss this later.

For example, the big thing "skipped" earlier was the return value…​ bind returns a UiBinding.Binding object when performing the bind. This object allows you to manipulate aspects of the binding specifically unbind a component and also manipulate auto commit for a specific binding.

Auto commit determines if a property is changed instantly or on commit. This is useful for a case where you have an "OK" button and want the changes to the UI to update the properties when "OK" is pressed (this might not matter if you keep different instances of the object). When autocommit is on (the default which you can change through setAutoCommit in the UiBinding) changes reflect instantly, when it’s off you need to explicitly call commit() or rollback() on the Binding class.

commit() applies the changes in the UI to the properties, rollback() restores the UI to the values from the properties object (useful for a "reset changes" button).

Binding also includes the ability to "unbind" this is important if you have a global object that’s bound to a UI that’s discarded. Binding might hold a hard reference to the UI and the property object might create a memory leak.

By using the disconnect() method in Binding you can separate the UI from the object and allow the GC to clean up.

Up until now this was pretty cool but if you looked at the UI construction code above you would see that it’s pretty full of boilerplate code. The thing about boilerplate is that it shows where automation can be applied, that’s the exact idea behind the magical "InstantUI" class. This means that the UI above can be generated using this code:

This still carries most of the flexibilities of the regular binding for example: you can still get a binding object using:

You might not have noticed this but in the previous verbose code you had lines like:

You might be surprised to know that this will still work seamlessly without doing anything, as would the picker component used to pick a date…​

The picker component implicitly works for date type properties, numeric constraints and numbers are implicitly used for number properties and check boxes are used for booleans.

However, how do you know to use an email constraint for the email property?

You’ve some special case defaults for some common property names, so if your property is named email it will use an email constraint by default. If it’s named url or password etc. It will do the "right thing" unless you explicitly state otherwise. You can customize the constraint for a specific property using something like:

This will override the defaults you’ve in place. The goal of this tool is to have sensible "magical" defaults that "work."