Export to ‘.tmat.h5’ format with Julia

Author

baptiste

Published

June 27, 2024

This document showcases a basic Julia script to export T-matrices in the .tmat.h5 HDF5 format. For illustration, we start by producing a dummy dataset. This minimal reproducible example file is available for download as a standalone script: export_julia.jl.

Mockup input data

Consistent with the other examples, we generate a 50x30x30 array of dummy data, which repeats 50 times (50 wavelengths) a matrix of 900 entries ranging from \(1+1i\) (first element, top left) to \(900 + 900i\) (last element, bottom right). Note the expectation of row-major ordering in HDF5. The 3x3 top-left block is

   1.0000 + 1.0000i   2.0000 + 2.0000i   3.0000 + 3.0000i ...
  31.0000 +31.0000i  32.0000 +32.0000i  33.0000 +33.0000i ...
  61.0000 +61.0000i  62.0000 +62.0000i  63.0000 +63.0000i ...
   ...               ...                ...               ...

For convenience, we can store compound objects as named tuples, which will then be mapped into HDF5 groups during export.

## mockup data
using Pkg, HDF5

# possibly multiple wavelengths
wavelength = collect(400:50:800)
Nl = length(wavelength)
Lmax = 3
qmax = 2*(Lmax*(Lmax+1)+Lmax)

# dummy 30x30 matrix values for each wavelength
# note the row-major ordering
tdata = transpose(reshape(collect(1:qmax^2) + collect(1:qmax^2) * 1im, (qmax,qmax)))
tmatrix = zeros(ComplexF64,(Nl,qmax,qmax))
for i=1:Nl
    tmatrix[i,:,:] = tdata
end

print(tmatrix[1,1:3,1:3])

l = zeros(Int64, qmax)
m = zeros(Int64, qmax)
s = Vector{String}(undef,qmax)
let
i=1
for li = 1:Lmax
    for mi = -li:li
        for si = ["electric", "magnetic"]
            l[i] = li
            m[i] = mi
            s[i] = si
            i = i+1
        end
    end
end
end

Saving to HDF5

Complex arrays are stored with r and i fields as usual. Note that Julia has column-major ordering, unlike HDF5, and we should therefore explicitly transpose the matrix before saving it, to avoid confusion if the data are to be read in another language. Another potential source of error is that the HDF5.jl library orders the array data differently, compared to h5py.

f = "aj.tmat.h5"
ver = Pkg.Operations.Context().env.manifest
h5ver = string(ver[findfirst(v->v.name == "HDF5", ver)].version)
software = "SMARTIES=1.1, julia=$(VERSION), HDF5.jl=$(h5ver)"

h5open(f, "w") do fid

    fid["vacuum_wavelength"] = wavelength
    attributes(fid["vacuum_wavelength"])["unit"] = "nm"

    # set = permutedims(dset, reverse(1:ndims(dset)))
    # https://juliaio.github.io/HDF5.jl/stable/#Language-interoperability-with-row-and-column-major-order-arrays
    fid["tmatrix"] = permutedims(tmatrix, reverse(1:ndims(tmatrix)))

    modes = create_group(fid, "modes") 
    modes["l"] = l
    modes["m"] = m
    modes["polarization"] = s

    embedding = create_group(fid, "embedding") 
    embedding["relative_permittivity"] = 1.33^2
    embedding["relative_permeability"] = 1.0
    attributes(embedding)["name"] = "H2O, Water"
    attributes(embedding)["keywords"] = "non-dispersive"

    sca_mat = create_group(fid, "scatterer/material") 
    sca_mat["relative_permittivity"] = repeat([-11.4 + 1.181im], Nl)
    sca_mat["relative_permeability"] = 1.0
    attributes(sca_mat)["name"] = "Au, Gold"
    attributes(sca_mat)["keywords"] = "dispersive, plasmonic"
    attributes(sca_mat)["reference"] = "Au from Raschke et al 10.1103/PhysRevB.86.235147"

    sca_geo = create_group(fid, "scatterer/geometry") 
    sca_geo["radiusxy"] = 20.0
    sca_geo["radiusz"] = 40.0
    attributes(sca_geo)["unit"] = "nm"
    attributes(sca_geo)["shape"] = "spheroid"
    attributes(sca_geo)["name"] = "homogeneous spheroid with symmetry axis z"

    mpar = create_group(fid, "computation/method_parameters") 
    mpar["Lmax"] = Lmax
    mpar["Ntheta"] = 100
    script = create_group(fid, "computation/files") 
    script["script"] = read("export_julia.jl", String)
    
    # write root attributes
    attributes(fid)["name"] = "Au prolate spheroid in water"
    attributes(fid)["description"] = "Computation using SMARTIES, a numerically robust EBCM implementation for spheroids"
    attributes(fid)["keywords"] = "gold, spheroid, ebcm, passive, reciprocal, czinfinity, mirrorxyz"    
    attributes(fid)["storage_format_version"] = "v0.01"
    
    # comp attributes
    attributes(fid["computation"])["method"] = "EBCM, Extended Boundary Condition Method"
    attributes(fid["computation"])["description"] = "Computation using SMARTIES, a numerically robust EBCM implementation for spheroids"
    attributes(fid["computation"])["name"] = "SMARTIES"
    attributes(fid["computation"])["software"] = software

end

Summary of the file:

<list>
├─data: <list>
│ ├─computation: <list>
│ │ ├─files: <list>
│ │ │ └─script: "## mockup data↵using Pkg, HDF5↵↵..."
│ │ └─method_parameters: <list>
│ │   ├─Lmax: 3
│ │   └─Ntheta: 100
│ ├─embedding: <list>
│ │ ├─relative_permeability: 1
│ │ └─relative_permittivity: 1.7689
│ ├─modes: <list>
│ │ ├─l<int [30]>: 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, ...
│ │ ├─m<int [30]>: -1, -1, 0, 0, 1, 1, -2, -2, -1, -1, ...
│ │ └─polarization<chr [30]>: "electric", "magnetic", "electric", "magnetic", "electric", "magnetic", "electric", "magnetic", "electric", "magnetic", ...
│ ├─scatterer: <list>
│ │ ├─geometry: <list>
│ │ │ ├─radiusxy: 20
│ │ │ └─radiusz: 40
│ │ └─material: <list>
│ │   ├─relative_permeability: 1
│ │   └─relative_permittivity<cpl [9]>: -11.4+1.181i, -11.4+1.181i, -11.4+1.181i, -11.4+1.181i, -11.4+1.181i, -11.4+1.181i, -11.4+1.181i, -11.4+1.181i, -11.4+1.181i
│ ├─tmatrix<cpl [8,100]>: 1+1i, 1+1i, 1+1i, 1+1i, 1+1i, 1+1i, 1+1i, 1+1i, 1+1i, 31+31i, ...
│ └─vacuum_wavelength<int [9]>: 400, 450, 500, 550, 600, 650, 700, 750, 800
└─attributes: <list>
  ├─root: <list>
  │ ├─description: "Computation using SMARTIES, a nu..."
  │ ├─keywords: "gold, spheroid, ebcm, passive, r..."
  │ ├─name: "Au prolate spheroid in water"
  │ └─storage_format_version: "v0.01"
  ├─vacuum_wavelength: <list>
  │ └─unit: "nm"
  ├─computation: <list>
  │ ├─description: "Computation using SMARTIES, a nu..."
  │ ├─method: "EBCM, Extended Boundary Conditio..."
  │ ├─name: "SMARTIES"
  │ └─software: "SMARTIES=1.1, julia=1.9.3, HDF5...."
  ├─embedding: <list>
  │ ├─keywords: "non-dispersive"
  │ └─name: "H2O, Water"
  ├─material: <list>
  │ ├─keywords: "dispersive, plasmonic"
  │ ├─name: "Au, Gold"
  │ └─reference: "Au from Raschke et al 10.1103/Ph..."
  └─geometry: <list>
    ├─name: "homogeneous spheroid with symmet..."
    ├─shape: "spheroid"
    └─unit: "nm"