The dataset used in this example notebook is from the Nakadake Sanroku Kiln Site Center in Japan. The data set is provided by Shinoto et al. under the CC-BY-4.0 license: DOI

Selecting a filter pipeline for a dataset

import afwizard

The goal of this notebook is to find the best fitting filter pipeline for a given dataset (or rather a suitably copped subset there of). If you want to manually create the filter pipeline from scratch, you should read the notebook on pipeline creation instead. Here, we assume that we want to leverage existing, community-contributed filter pipelines. In a first step, we select a number of filter pipeline candidates from filter libraries. Details about how to register additional filter libraries, can be found in the notebook on filter libraries. We use the select_pipelines_from_library function, which allows us to select any number of filter by keeping CTRL pressed while clicking additional filters. In the left most column, we see filtering criterions we can use to access the filter pipelines in our filtering libraries. The middle column contains a list of filter pipelines that match the given criteria. Clicking one of these will select it and show its metadata in the third column. Multiple filters can be selected by keeping CTRL pressed while clicking additional filters.

pipelines = afwizard.select_pipelines_from_library()

---------------------------------------------------------------------------
AFwizardError                             Traceback (most recent call last)
Cell In[2], line 1
----> 1 pipelines = afwizard.select_pipelines_from_library()

File ~/checkouts/readthedocs.org/user_builds/afwizard/conda/stable/lib/python3.11/site-packages/afwizard/apps.py:1223, in select_pipelines_from_library()
   1211 def select_pipelines_from_library():
   1212     """The Jupyter UI to select filtering pipelines from libraries.
   1213
   1214     The use of this UI is described in detail in `the notebook on filtering libraries`_.
   (...)
   1220     :rtype: afwizard.filter.Pipeline
   1221     """
-> 1223     return select_pipeline_from_library(multiple=True)

File ~/checkouts/readthedocs.org/user_builds/afwizard/conda/stable/lib/python3.11/site-packages/afwizard/apps.py:1045, in select_pipeline_from_library(multiple)
   1040 # Collect checkboxes in the selection menu
   1041 library_checkboxes = [
   1042     ipywidgets.Checkbox(value=True, description=library_name(lib), indent=False)
   1043     for lib in get_filter_libraries()
   1044 ]
-> 1045 backend_checkboxes = {
   1046     name: ipywidgets.Checkbox(value=cls.enabled(), description=name, indent=False)
   1047     for name, cls in Filter._filter_impls.items()
   1048     if Filter._filter_is_backend[name]
   1049 }
   1051 # Extract all authors that contributed to the filter libraries
   1052 def get_author(f):

File ~/checkouts/readthedocs.org/user_builds/afwizard/conda/stable/lib/python3.11/site-packages/afwizard/apps.py:1046, in <dictcomp>(.0)
   1040 # Collect checkboxes in the selection menu
   1041 library_checkboxes = [
   1042     ipywidgets.Checkbox(value=True, description=library_name(lib), indent=False)
   1043     for lib in get_filter_libraries()
   1044 ]
   1045 backend_checkboxes = {
-> 1046     name: ipywidgets.Checkbox(value=cls.enabled(), description=name, indent=False)
   1047     for name, cls in Filter._filter_impls.items()
   1048     if Filter._filter_is_backend[name]
   1049 }
   1051 # Extract all authors that contributed to the filter libraries
   1052 def get_author(f):

File ~/checkouts/readthedocs.org/user_builds/afwizard/conda/stable/lib/python3.11/site-packages/afwizard/lastools.py:139, in LASToolsFilter.enabled(cls)
    137 @classmethod
    138 def enabled(cls):
--> 139     return lastools_is_present()

File ~/checkouts/readthedocs.org/user_builds/afwizard/conda/stable/lib/python3.11/site-packages/afwizard/lastools.py:69, in lastools_is_present()
     66         return False
     68 # We return True iff a prefix was set
---> 69 return get_lastools_directory() is not None

File ~/checkouts/readthedocs.org/user_builds/afwizard/conda/stable/lib/python3.11/site-packages/afwizard/lastools.py:55, in get_lastools_directory()
     53     dir = os.environ.get("LASTOOLS_DIR", None)
     54     if dir is not None:
---> 55         set_lastools_directory(dir)
     57 return _lastools_directory

File ~/checkouts/readthedocs.org/user_builds/afwizard/conda/stable/lib/python3.11/site-packages/afwizard/lastools.py:44, in set_lastools_directory(dir)
     42 except AFwizardError as e:
     43     _lastools_directory = None
---> 44     raise e

File ~/checkouts/readthedocs.org/user_builds/afwizard/conda/stable/lib/python3.11/site-packages/afwizard/lastools.py:41, in set_lastools_directory(dir)
     38 if dir is not None:
     39     try:
     40         # If this throws, we show a meaningful error where we looked for LASTools
---> 41         lasground_executable(base=dir)
     42     except AFwizardError as e:
     43         _lastools_directory = None

File ~/checkouts/readthedocs.org/user_builds/afwizard/conda/stable/lib/python3.11/site-packages/afwizard/lastools.py:84, in lasground_executable(base)
     82 fullpath = os.path.join(base, "bin", execname)
     83 if not os.path.exists(fullpath):
---> 84     raise AFwizardError(f"Executable {fullpath} was not found!")
     86 return fullpath

AFwizardError: Executable /home/docs/checkouts/readthedocs.org/user_builds/afwizard/checkouts/latest/LAStools/bin/lasground_new64.exe was not found!

Next thing to do is to load a dataset as described in Working with datasets. Again, it is best to restrict the sample size to a reasonable size (e.g. by using the restrict method on the dataset) to allow a truely interactive exploration process.

dataset = afwizard.DataSet(
    filename="nkd_pcl_epsg6670.laz", spatial_reference="EPSG:6670"
)

We can then create a comparison of ground point filtering results with the goal of choosing the most fitting pipeline. Each tab shows the filtering results of one of the selected pipelines. In the left column, we can fine tune visualization and rasterization options while on the right hand side, we can fine tune the filter. The configuration options shown here have been introduced by the author of the filter pipeline for you to fine tune the results:

best = afwizard.select_best_pipeline(dataset, pipelines=pipelines)

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[4], line 1
----> 1 best = afwizard.select_best_pipeline(dataset, pipelines=pipelines)

NameError: name 'pipelines' is not defined

The newly create filter pipeline object best has the end user configuration specified in the right column baked into the filter. This means that the resulting filter is to some extent a dataset-specific specialization of the general purpose filter pipeline. To distinguish such specialized filter from more general ones, it is useful to save it into a separate filtering library as outlined in the notebook on filtering libraries, e.g. by using set_current_filter_library to set up a filtering library for your current project before saving any filters:

afwizard.set_current_filter_library(
    "projectx", create_dirs=True, name="Filters for projext X"
)

afwizard.save_filter(best, "bestfilter.json")

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[6], line 1
----> 1 afwizard.save_filter(best, "bestfilter.json")

NameError: name 'best' is not defined