|
Research
Active Projects
|
eCNN: Embedded
Convolutional Network Processor
Convolutional networks, or CNN, recently have made
tremendous progress not only on artificial intelligence tasks,
such as object recognition and classification, but also on image
processing applications, including denoising and
super-resolution. It is highly potential to solve all these
vision tasks in one single CNN processor; however, the extremely
demanding complexity defies its usage on embedded devices so far. In this
project, we aim to propose a series of novel CNN processors to achieve
real-time, low-power, and high-resolution vision tasks. This
project is supported by
MOST (Semiconductor
Moonshot Project). [eCNN, MICRO'19][ERNet,
ICASSP'20][RingCNN, ISCA'21][AICAS'21][ESSCIRC'22] |
|
Advanced
Embedded Computing
In this project, we aim to push CNN acceleration to an advanced
level: 8K UHD applications with novel model features, such as
video processing, instance normalization, and channel attention.
To achieve high energy efficency and high computing performance
simultaneously, we apply a holistic approach to jointly optimize
hardware-oriented models, energy-efficient architecture, and
16nm chip implementation. This project is supported by
MOST and
TSMC.
[SiPS'21][VISTA, ISSCC'23] |
Completed Projects
|
Factored
Light-Field Display
Light-field display is an emerging method for true 3D display
devices, but the bottleneck for commercialization is its
low-resolution nature. Factored display is a technique to
increase the resolution by using multiple LCD layers and complex
light-field factorization. In this project, we will build a
prototype factored display to study how its system parameters
affect visual quality and will also develop high-quality and
cost-effective factorization algorithms for real-time and
low-power implementation. This project was supported by
Raydium. [ISSCC'22] |
|
Empirical Bayesian
Neighborhood
Range-weighted neighborhood formulations are useful and popular
for their edge-preserving property and simplicity, but they are
mostly proposed as intuitive tools. This project studies how to
formulate these problems in empirical Bayesian ways, instead of
intuitive ways. It also aims to optimize the essential bandwidth
parameters
by model fitting to empirical distributions, instead of by
heuristic setting. The idea can also be extended to build Markov
Random Fields for stereo matching. This project was supported by
MOST.
Webpages:
[denoise, CVPR'15/TIP],
[fast fitting,
SPL'16], [stereo
matching, ICCV'17/TPAMI] |
|
Phase-Based
Signal Processor
Complex steerable pyramid (CSP) can decompose signals into
pixel-wise phase information, and manipulating the phase can
easily create position-shift effect. Therefore, it can provide
efficient frame interpolation, view synthesis, and even digital
refocusing in one single processor. In this project, we aim to
design and implement a high-performance phase-based signal
processor for supporting such diverse applications in one
unified framework. This project was supported by
MOST.
[ICASSP'19][ICASSP'20] |
|
Image
Deblurring for FORMOSAT-5
FORMOSAT-5 is the first remote
sensing satellite entirely made in Taiwan and aims to build up
Taiwan's self-reliant space tecnology. However, it was found to
have defocus issues in its optical system after launch. In this
project, we collaborate with NSPO researchers to deblur its
remote sensing images. In contrast with conventional
algorithms which cause over-smoothness, we aim to recover
original texture and maintain vivid realisticity based on the
specific priors of its blur kernel. This project was supported by
NSPO.
[ACRS'18] |
|
Satellite
Light-Field Processor
We define a satellite light field by one full-resolution center
view and four half-resolution surrounding views. This setting
provides cost-effective but high-quality light-field systems. In
this project, we aim to build powerful chips to bring
low-power computational photography applications in real time.
One application is high-quality depth esitmation (stereo
matching) at Full HD to 4K Ultra HD 30fps, and the
other one is physically realistic refocusing for Full HD
videos.
This project was supported by
Novatek
and
MOST.
[ISSCC'15][ASSCC'18][VLSIC'19][OJ-SSCS'23] |
|
DragonFly
Lens-array cameras provide the
opportunity to have high SNR image quality on thin mobile
devices by using DSLR-level sensors. They also can bring us
novel light-field applications, such as digital refocus and
natural matting. This project focuses on algorithm developement
for disparity-based refocusing, denoising, and matting. To show how
this can change the way we capture videos, prototype
lens-array cameras, DragonFly, and a real-time demo platform
will be established. This project was supported by
MOST.
Webpages:
[refocus
& DragonFly, ICASSP'15], [block-based
refocusing, TIP], [stereo
matching, ICCV'17/TPAMI] |
|
Super-Vision
Humans can only have rough perception
for early-vision information, such as object distance, size, and
speed. This project targets to develop a low-level vision system, which is
based on stereo imaging, for acquiring such early-vision
information but in high precision. It will not only help people
understand the physical structure of interested objects faster
and more precisely, but also can assist and improve the
efficiency of high-level computer vision applications. This
project was supported by
Novatek.
|
|
|
|
|