METHOD FOR DEPTH-DETAIL MAP (DDM) CREATION FOR DETAIL AMPLIFICATION OF DEPTH MAP IN ALPHAS USING DEPTH MAP AND RGB IMAGE.

Abstract

ABSTRACT METHOD FOR DEPTH-DETAIL MAP (DDM) CREATION FOR DETAIL AMPLIFICATION OF DEPTH MAP IN ALPHAS USING DEPTH MAP AND RGB IMAGE. Present invention discloses a process(100) developed to generate depthdetail map(DDM)(102 b) from depth map and a RGB image as input for creating a more realistic depth map with detail amplification. In this approach the depth map(102 c) generated from the RGB image is taken as primary input, this depth map is then superimposed with colour ramped interpolated greyscale image(105 b)(obtained from RGB image) as secondary input to add finer details to the depth map hence increasing its granularity and adding the fine details(103 a). The grey tones of the greyscale image(105 a) are adjusted using interpolation techniques so that they are normalised and smooth all over the image, this creates a smooth curve in high gradient grey areas of the depth detail map(DDM)(102 b). The depth map(102 c) which is generated from image has the height information but lack fine-granulated details(103b). DDM(102 b, 103 a) has more fine and granulated details along with the height information and looked more realistic(103a) as compared to the depth map(103 b) alone for creating alphas(103c), used in 2D image to 3D conversions(103 a).

Specification

Description:FIELD OF INVENTION
This invention relates to the field of image processing in computer
imaging. It introduces a new and efficient method(100) of increasing the
granularity, precision and accuracy of depth maps created from single
monocular image by generating the depth-detail map(DDM)(106, 103a).
BACKGROUND OF INVENTION
Depth maps are a critical component in the fields of image processing,
computer vision, providing spatial information that facilitates the
conversion of 2D images into 3D representations. A depth map is a
greyscale image where each pixel's intensity corresponds to the distance
from the camera to the scene's objects, effectively encoding depth
information. Depth maps are extensively used in various applications,
including 3D reconstruction, image-based rendering, photogrammetry, and
motion tracking, where accurate depth information is essential for creating
realistic 3D models from 2D imagery.
In the context of 2D-to-3D conversion, depth maps are commonly utilized
as alpha channels or per-pixel depth data to inform the transformation of a
flat, two-dimensional image into a three-dimensional space. By using
depth values, these maps guide the extrusion or warping of 2D images,
simulating the visual effects of depth layering, occlusion, and perspective.
This approach is frequently applied in generating 3D meshes. The use of
depth maps for 2D-to-3D conversion enables the creation of 3D objects or
scenes from a single image, facilitating applications in 3D object modelling
from images, digital content creation.

Despite their utility in 2D-to-3D conversion, depth maps often lack the finegrained details required for high-precision alpha representations. The
pixel-based depth values typically provide a coarse approximation of
spatial distances, which can lead to inaccuracies in modelling intricate
surface features, subtle depth variations. This limitation reduces the
overall quality of 3D reconstructions, particularly in complex scenes where
minute details are critical for realism and accuracy.
To overcome the limitations of depth maps in capturing fine details, we
have proposed this method.
SUMMARY OF INVENTION
Present invention discloses a process(100) developed to generate depthdetail map(DDM)(106) from depth map and a RGB image as input for
creating a more realistic depth map with detail amplification. In this
approach the depth map(102 c) generated from the RGB image taken as
primary input, this depth map is then superimposed with colour ramped
interpolated greyscale image(105b)(obtained from RGB image) as
secondary input to add finer details to the depth map hence increasing its
granularity and adding the fine details(103 a). The grey tones of the
greyscale image(105 a) are adjusted using interpolation techniques so that
they are normalised and smooth all over the image, this creates a smooth
curve in high gradient grey areas of the depth detail map(DDM)(102 b).
The depth map(102 c) which is generated from image has the height
information but lack fine-granulated details(103 b). DDM(102b, 103a) has
more fine and granulated details along with the height information and
looked more realistic(103a) as compared to the depth map(103 b) alone
for creating alphas(103c), used in 2D image to 3D conversions(103 a).

BRIEF DESCRIPTION OF DIAGRAMS
Fig 100:- Flow Diagram of Depth-Detail Map(DDM) creation.
Fig 101:- Colour ramp Interpolation
Fig 102:-
102(a) = RGB image used for testing
102(b) = DDM (Depth Detail Map)
102(c) = Depth map
Fig 103:-
103(a) = 3D from 2D using Alpha from DDM map (granulated and
fine details are visible)
103(b) = 3D from 2D using Alpha from Depth map (less details
compared to DDM)
103(c) = DDM vs Depth
Fig 104:-
104(a) = Pre DDM creation step (Depth map)
104(b) = Pre DDM creation step (Greyscale image)
104(c) = Post DDM creation step (DDM Depth-detail map)
Fig 105:-
105(a) = Greyscale image pre colour ramp interpolation
105(b) = Greyscale image post colour ramp interpolation
Fig 106:- Depth Detail Map creation(DDM)

DETAILED DESCRIPTION OF INVENTION
Depth map plays an important role in alpha creation, depth map is created
from single monocular image using the marigold depth estimation/ zoe
depth technique which contains the height/ space/ Z value from an RGB
image which gives the depth mapping now to add more granularity and
fine details a method is introduced using the greyscale image(from the
RGB image) and by using the value mapping, interpolation and
superimposition techniques depth-detail map is created.(100)
Depth map creation:
The Marigold Depth Estimation Technique is used for creating depth maps
from a single 2D image. The core principle behind this technique is to
estimate pixel-wise depth values by leveraging both global image features
and local spatial information, typically in the form of multi-scale
convolutional neural networks (CNNs).
The process begins by extracting the feature representations of the input
image I, using a CNN. This is typically done by applying multiple layers of
convolutions to capture both high-level contextual and low-level edge
details:
F = CNN(I) (1)
where F represents the feature map of the image.
Next, the depth D(x, y) for each pixel (x, y) is predicted using a regression
model that maps the feature map F to depth values:
D(x, y) =Regressor(F(x, y)) (2)
where Regressor is a function learned during training to map features to
depth values.

Black and white image creation(Greyscale):
The process of converting a coloured image(102 a) to greyscale image
(105 a) involves transforming the RGB(red, green, blue) colour values to
greyscale representation. This is achieved by combining the colour
channels. Simple grey conversion technique is used in this case this
method involves calculating the luminance(brightness) of each pixel by
averaging the RGB values:
Luminance(V) = 0.299(R) + 0.587(G) + 0.114(B) (3)
Here R,G,B are the RGB colour channel value ranging from 0(minium) to
255(maximum) intensity. The weighing is based on human visual
perception where green is more visible to eyes than red, blue. This formula
helps create a balanced grey scale image(105 a) where brightness more
closely resembles with real world.
Colour ramp with 2 stops:
The colour ramp(101) is used to normalise the grey tones of black and
white image so as to get smooth transition and blend with depth map.
The colour ramp has mainly 2 steps(101):
1. Value mapping
2. Interpolation
Value mapping:- The value mapping typically works by mapping an
input value typically a greyscale value to a corresponding position
along gradient of colours, defined across colour ramp which starts
from 0(white) to 1(black) and has all grey variations in between.
Input range - Input value is grey scale value, this value acts as coordinate along the colour ramp.

Mapping along colour ramp involves 2 stops:-
Stop 1 is 0- mapped to beginning of colour ramp(white in this case)
Stop 2 is 1- mapped to end of colour ramp(black in this case)
when input falls between two colour stops, an intermediate colour is
calculated by interpolating between the colours at this point on
colour ramp.
Interpolation: - Interpolation is the key process that determine how
colour blends between stops(black, white)(105 b). Linear
interpolation technique is used here, mathematically if their are two
colours C1 at position 1(white in this case) and C2 at position 2
(black in this case). Then colour Cn at point n between them will be
given by formula
Cn
= C1
+ (C2−C1
)⋅t (4)
Here t is normalised distance between points 1 and 2 called as
interpolation factor.
Example:
Given a greyscale image(converted from RGB image)(105 a) and
considering one pixel at point k as Ck(ranges from 0-255 because of
greyscale value range) on the greyscale image and having two
control points C1 at 0(white in this case) and C2 at 1(black in this
case) such that Ck is C1<(Ck/255)< C2
Ck is here after considered as Ck/255 and lies in range of 0 to 1
then after interpolation new output pixel colour Cnew is given by

Cnew = C1+
Ck−C1
C2−C1
⋅(C2−C1) (5)
where
Ck−C1
C2−C1
is the interpolation factor.
Ck is the input colour pixel value in greyscale image at point k.
Cnew is the new output colour pixel value at point k.
K is point in image
hence we get a colour ramped interpolated greyscale image(105b)
at end of this step.
Depth - Detail Map Creation Superimposition Step:
Now considering two maps to be superimposed
1. Depth map (102c)
2. Colour ramped Interpolated greyscale image. (105b)
In this step(106) for superimposition depth map(102 c) is considered as
bottom layer and colour ramped interpolated greyscale image(105 b) as
top layer, now both these layers are combined based on their alpha values
of the respective images/maps, alpha composting formula is used this
creates a super imposition by weighing each pixel colour according to
transparency.
Suppose
Cf = (Rf , Gf , Bf , Af) colours and alpha of top layer pixel
Cb = (Rb, Gb, Bb, Ab) colours and alpha of bottom layer pixel
Then the resulting pixels output of the Depth-Detail map(DDM)(102b)
denoted by Cddm by combining both the bottom and top layers will be given
by:

Cddm = (Addm, Rddm, Gddm, Bddm)
where
Addm = Af + Ab⋅(1− Af) (6)
Rddm =
Rf ∗ Af +Rb⋅ Ab⋅(1− Af)
Rddm
(7)
Gddm =
Gf ∗ Af +Gb⋅ Ab⋅(1− Af)
Gddm
(8)
Bddm =
Bf ∗ Af +Bb⋅ Ab⋅(1− Af)
Bddm
(9)
Top layer contribution = Rf⋅Af
The top layer is weighted by its opacity
Bottom layer contribution = Rb⋅Ab⋅(1− Af)
The bottom layer is weighted by its opacity, further reduced by the
transparency of the top layer.
Division by Addm = This normalises the output, ensuring the colour remains
within the range of 0 to 1
For Example:
Foreground: Cf=(0.2,0.9,0.2,0.6)
Background: Cb=(0.4,0.7,0.3,1.0)

Then:
Alpha Addm= 0.6 + 1.0 (1−0.6) = 1.0 ⋅
Red Rddm =
0.2⋅0.6+0.4 ⋅1.0⋅(1−0.6)
1.0 = 0.28
Green Gddm =
0.9⋅0.6+0.7⋅1.0⋅(1−0.6)
1.0 = 0.82
Blue Bddm =
0.2⋅0.6+0.3⋅1.0⋅(1−0.6)
1.0 = 0.24
So, the output colour Cddm=(0.28,0.82,0.24,1.0), which is a combination of
the foreground and background colours, blended based on their
transparency.
Hence the outputted values Addm, Rddm, Gddm, Bddm become the pixel values
of the Depth Detail Map(DDM)(102b), and hence DDM is created. Which
can be used to create 3D(103 a) from 2D image using ddm(102 b) as
alpha.
Blur:
A fast guassian blur is applied on the depth detail map to further smooth
transition between the grey colour gradients.
Hence we get the final Depth-Detail Map(DDM)(102 b) created with super
fine minute details from the greyscale image and the height data from the
depth map making the DDM more realistic and precise with real world. , Claims:WE CLAIM
1. A method(100) for depth-detail map creation(102 b) for detail
amplification of depth map in computer imaging consisting the
steps of-
• Implementing the depth-detail map creation step(106) where
the depth map(102 c) and interpolated greyscale image(105
b) is super imposed depending on their alpha values.
• Using the depth-detail map(102 b) as alpha for creating a 3D
model(103 a) from a 2D image.
• Creating a smooth gradient DDM using the fast-guassian
blur technique.
• Using value mapping and Interpolation techniques on
greyscale image to normalise the grey tones of the greyscale
image specifically using two stop points approach at white(0)
and black(1).
• Using the interpolated greyscale image(105 b) and depth
map and superimposing(106) them to produce depth-detail
map (DDM)(102 b, 103 a).
2. As mentioned in Claim 1, Implementing the depth-detail map
creation step(106) where the depth map(102 c) and
interpolated greyscale image(105 b) is super imposed
depending on their alpha values to create depth-detail
map(DDM)(102 b, 103 a).
3. As mentioned in Claim 1, using two stop points having
white(0) and black(1) for interpolating the greyscale
Image(101).

Documents