Mask images
Last compiled: 26 September 2023
mask_images.RmdHere we’ll have a quick look at how you can mask H&E images with
semla. Masking means removing the background from the
tissue section and is mostly useful for aesthetic purposes.
Load data
First we need to load some 10x Visium data. here we’ll use a mouse brain tissue dataset and a mouse colon dataset that are shipped with semla.
# Load data
se_mbrain <- readRDS(file = system.file("extdata",
"mousebrain/se_mbrain",
package = "semla"))
se_mbrain$sample_id <- "mousebrain"
se_mcolon <- readRDS(file = system.file("extdata",
"mousecolon/se_mcolon",
package = "semla"))
se_mcolon$sample_id <- "mousecolon"
se_merged <- MergeSTData(se_mbrain, se_mcolon) |>
LoadImages()When we plot the H&E images, we can see that the entire capture area is shown, including the fiducials (the dots that marks the edges of the capture area).
ImagePlot(se_merged)
Mask images
MaskImages makes it possible to remove the
background:
se_merged <- se_merged |>
MaskImages()
ImagePlot(se_merged)
When masking H&E images in a Seurat object created
with semla, the “raw” image is replaced, meaning that plot
functions such as MapFeatures and MapLabels
will now use the masked image instead.
MapFeatures(se_merged, features = c("Th", "Il22ra2"), image_use = "raw",
override_plot_dims = TRUE, colors = RColorBrewer::brewer.pal(n = 9, name = "Spectral") |> rev())
If you want to use the original H&E images, you can simply reload
them with LoadImages.
# Reload images from source files
se_merged <- LoadImages(se_merged)Notes about H&E masking
Masking is not always a trivial task and MaskImages
might fail, in particular when faced with one of the following
issues:
presence of staining artefacts
when using other stains than H&E
presence of bubbles or other types of speckles/dust
tissues with low contrast to background, e.g. adipose tissue
if the image is loaded in high resolution
Custom masking (advanced)
If MaskImages fails, it is possible to mask the images
manually, but this requires some knowledge about image processing. Below
is a simple example of how one can mask the mouse brain tissue section
using the magick R package:
# Fetch H&E rasters
mcolon_rasters <- se_mcolon |> LoadImages() |> GetImages()
# Load image as a magick-image object
image <- image_read(mcolon_rasters[[1]])
# Convert image to CMYK colorspace and extract the magenta channel
im_magenta <- image |>
image_convert(colorspace = "cmyk") |>
image_channel(channel = "Magenta")
# Add blur effect to image and threshold image
im_threshold <- im_magenta |>
image_blur(sigma = 2) |>
image_threshold(type = "black", threshold = "20%") |>
image_threshold(type = "white", threshold = "20%")
# Mask H&E by combining H&E image with mask
mask <- im_threshold |>
image_transparent(color = "black")
im_composite <- image_composite(mask, image)
# Plot images
par(mfrow = c(2, 2), mar = c(0, 0, 0, 0))
image |> as.raster() |> plot()
title("H&E image", line = -2)
im_magenta |> as.raster() |> plot()
title("Magenta color channel", col.main = "white", line = -2)
im_threshold |> as.raster() |> plot()
title("Image mask", col.main = "white", line = -2)
im_composite |> as.raster() |> plot()
title("Masked H&E image", line = -2)
The results are not perfect because there are still a few speckles in the background and some parts of the tissue section are masked. But even a simple approach like this can give decent results!
The processing was done using the R package magick and
with this package you should be able to manipulate images to get pretty
much any result you want. The package vignette is a good resource to get
started: magick
intro
Once you have masked the image, you can convert it back to a
raster object and place it into your Seurat
object. Now we are only working with 1 tissue section, but if you have
multiple tissue sections you need to make sure that the list of
raster objects contains 1 image per sample in the correct
order.
Also, you cannot adjust the dimensions of the image!!! They have to have exactly the same dimensions as the images that you started with otherwise, the spots will no longer be aligned properly.
se_mcolon@tools$Staffli@rasterlists$raw <- list(as.raster(im_composite))
MapFeatures(se_mcolon, features = "nFeature_Spatial", image_use = "raw",
pt_alpha = 0.5, pt_size = 1.5)
Package version
-
semla: 1.1.6
Session info
## R version 4.2.3 (2023-03-15)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS Big Sur ... 10.16
##
## Matrix products: default
## BLAS: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib
##
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
##
## attached base packages:
## [1] stats graphics grDevices datasets utils methods base
##
## other attached packages:
## [1] magick_2.7.4 semla_1.1.6 ggplot2_3.4.3 dplyr_1.1.3
## [5] SeuratObject_4.1.3 Seurat_4.3.0.1
##
## loaded via a namespace (and not attached):
## [1] Rtsne_0.16 colorspace_2.1-0 deldir_1.0-9
## [4] ellipsis_0.3.2 ggridges_0.5.4 rprojroot_2.0.3
## [7] fs_1.6.3 spatstat.data_3.0-1 farver_2.1.1
## [10] leiden_0.4.3 listenv_0.9.0 ggrepel_0.9.3
## [13] fansi_1.0.4 codetools_0.2-19 splines_4.2.3
## [16] cachem_1.0.8 knitr_1.43 zeallot_0.1.0
## [19] polyclip_1.10-4 jsonlite_1.8.7 ica_1.0-3
## [22] cluster_2.1.4 png_0.1-8 uwot_0.1.16
## [25] spatstat.sparse_3.0-2 shiny_1.7.5 sctransform_0.3.5
## [28] BiocManager_1.30.22 compiler_4.2.3 httr_1.4.7
## [31] Matrix_1.5-1 fastmap_1.1.1 lazyeval_0.2.2
## [34] cli_3.6.1 later_1.3.1 htmltools_0.5.6
## [37] tools_4.2.3 igraph_1.5.1 gtable_0.3.4
## [40] glue_1.6.2 RANN_2.6.1 reshape2_1.4.4
## [43] Rcpp_1.0.11 scattermore_1.2 jquerylib_0.1.4
## [46] pkgdown_2.0.7 vctrs_0.6.3 nlme_3.1-163
## [49] spatstat.explore_3.2-1 progressr_0.14.0 lmtest_0.9-40
## [52] spatstat.random_3.1-5 xfun_0.40 stringr_1.5.0
## [55] globals_0.16.2 mime_0.12 miniUI_0.1.1.1
## [58] lifecycle_1.0.3 irlba_2.3.5.1 renv_1.0.2
## [61] goftest_1.2-3 future_1.33.0 MASS_7.3-60
## [64] zoo_1.8-12 scales_1.2.1 spatstat.utils_3.0-3
## [67] ragg_1.2.5 promises_1.2.1 parallel_4.2.3
## [70] RColorBrewer_1.1-3 yaml_2.3.7 memoise_2.0.1
## [73] reticulate_1.31 pbapply_1.7-2 gridExtra_2.3
## [76] sass_0.4.7 stringi_1.7.12 highr_0.10
## [79] desc_1.4.2 rlang_1.1.1 pkgconfig_2.0.3
## [82] systemfonts_1.0.4 matrixStats_1.0.0 evaluate_0.21
## [85] lattice_0.21-8 tensor_1.5 ROCR_1.0-11
## [88] purrr_1.0.2 labeling_0.4.3 patchwork_1.1.3
## [91] htmlwidgets_1.6.2 cowplot_1.1.1 tidyselect_1.2.0
## [94] parallelly_1.36.0 RcppAnnoy_0.0.21 plyr_1.8.8
## [97] magrittr_2.0.3 R6_2.5.1 generics_0.1.3
## [100] withr_2.5.0 pillar_1.9.0 fitdistrplus_1.1-11
## [103] abind_1.4-5 survival_3.5-7 sp_2.0-0
## [106] tibble_3.2.1 future.apply_1.11.0 KernSmooth_2.23-22
## [109] utf8_1.2.3 spatstat.geom_3.2-4 plotly_4.10.2
## [112] rmarkdown_2.24 grid_4.2.3 data.table_1.14.8
## [115] forcats_1.0.0 digest_0.6.33 xtable_1.8-4
## [118] dbscan_1.1-11 tidyr_1.3.0 httpuv_1.6.11
## [121] textshaping_0.3.6 munsell_0.5.0 viridisLite_0.4.2
## [124] bslib_0.5.1 shinyjs_2.1.0