通過(guò)計(jì)算的沉浸式展示

COMPUTATIONAL

IMMERSIVEDISPLAYSDANIEL

E.

NOVY,

BFA,

MAS

H

A

O

S

H

U

A

I

2

0

1

3

/

9

/

2

2DANIEL

E.

NOVY×Dan

Novy

(also

known

as

NovySan)

discovered

3D

animation

and

modeling

and

virtual

reality

applications

as

astudent

at

the

University

of

Illinois,

where

he

received

a

BFA

in

Theatre,

with

a

concentration

in

Acting

andDirecting,

and

an

MA

in

Theatre

History,

with

a

double

emphasis

in

the

technical

history

of

the

theatre

andshamanic

ritual

performance

in

pre-agrarian

societies.

(Ask

him

about

Wet

Money

in

the

Restoration

theatre,

butdon’t

get

him

started

on

the

rise

of

agriculture).×Sixteen

years

ago,

he

parlayed

his

independent

studies

with

Video

Toaster

and

the

earliest

incarnations

ofLightwave

into

a

career

in

visual

effects.

Today,

he

is

an

Emmy

and

Visual

Effects

Society

award

winningtechnical

supervisor

and

compositor;

chair

of

the

Visual

Effects

Society’s

Technology

Committee;

and

technicalblogger

for

the

VES.

Driven

by

curiosity

as

much

as

necessity,

and

spurred

on

by

the

frontier

atmosphere

of

the

early

digital

effects

industry,

he

has

developed

unique

experience

in

the

creation

and

supervision

of

3Danimation

and

modeling,

2D

compositing

and

match-moving,

fluid

simulation,

and

physical

fabrication

of

practicaleffects.

Eschewing

specialization,

he

possesses

additional

expertise

in

system

and

network

administration,clustered

storage

solutions,

database

frontend

development,

project

management,

and

pipeline

and

workflowmanagement

and

development.×

Inadditionto

hiscontinuing

work

in

visualeffects,

NovySan

has

also

branched

outinto

theproductionoftransmedia

andIMmedia,

in

aneffort

to

break

throughthelimits

of

the

mediated

experience.

His

goalis

to

decrease

the

alienation

fostered

bytraditional,

passive

media

consumption,

andincrease

social

interaction

throughtransparent,

interconnected

and

fluid

media,tocreate

an

enriched,active

and

inspiredimmediate

experience.×

NovySan

is

currently

a

PhD

student

at

theMIT

Media

Lab

building

computer

systemsyou

controlwith

your

brain.ABSTRACT×

Immersion

is

an

oft-quoted

but

ill-defined

term

used

to

describe

a

viewer

orparticipant’s

sense

of

engagement

with

a

visual

display

system

or

participatorymedia.

Traditionally,

advances

in

immersive

quality

came

at

the

high

price

of

ever-escalating

hardware

requirements

and

computational

budgets.×

沉浸感是一個(gè)經(jīng)常被引用，但含糊不清的術(shù)語(yǔ)，用來(lái)形容一個(gè)觀眾或參與者在一個(gè)視覺(jué)展示系統(tǒng)或媒體中的參與感。一般情況下，隨著硬件條件的不斷升級(jí)和計(jì)算成本的不斷提高，沉浸質(zhì)量也隨之提高?！?/p>

但這不是絕對(duì)的。THREE

SYSTEMS×

Infinity-by-Nine×

Narratarium× 3D

Telepresence

ChairINTRODUCTION×

Why

less

is

more×

Recently,therehavebeenavariety

of

efforts

to

increase

the

immersive

qualityof

visual

displays

bymanipulating

the

size

of

the

display,

as

with

large-format

televisions

and

IMAX

screens,

or

by

simulatingthree-dimensionalenvironmentsthroughthe

use

of

stereoscopicprojection.Whilethese

systemsattempttoincrease

a

viewer’s

feeling

of

being

present

in

the

spacedepicted

by

the

media,mostlackthe

ability

toadapt

dynamically

to

the

display

environment

and/or

the

narrative

content.

These

forms

are

still

guilty

ofpresenting

merelyaboundedfrustum’sworthofinformation

terminatingin

the

lens

of

the

eye,

with

nofundamentaladvances

to

the

form

or

new

affordances

presented

with

the

content.× A

song

played

louder

is

still

the

same

song;

scale

does

not

equal

innovation.×

放的再響的歌兒也是同一首歌，加大規(guī)模不等于創(chuàng)新?！?/p>

Immersiondoesnot

require

increased

screenreal

estate;

it

requires

additionalwaysof

deepening

theengagement

andattentionof

the

user.×

沉浸感并不要求提高屏幕的真實(shí)尺寸，它要能提供額外的方式來(lái)加強(qiáng)使用者的代入感和提高注意力×

Infinity-by-Nine

allows

us

to

simulate

a

full-scale

environment

within

a

limited

space,such

as

a

home

theater.×

有限空間內(nèi)的環(huán)境光模擬。×

The

Narratarium

is

a

context-aware,

360-degree,

catadiotropic（catadioptric）projection

system

that

enhances

a

user’s

interaction

with

a

story,

game,

or

otherobject.×

利用反射折射的360°投影系統(tǒng)，加強(qiáng)用戶與故事游戲等的交互體驗(yàn)?！?/p>

The

3D

Telepresence

Chair

adds

an

autostereoscopic

3D

display,

which

is

visiblewithout

special

glasses

or

other

equipment,

to

traditional

teleconferencing.×

3D遠(yuǎn)程會(huì)議椅BACKGROUND×

Helmet-mounted

displays

and

pinch

glove

interfaces

allowed

VR

participants

to

shutout

the

outside

world

completely

for

the

first

time

and

replace

it

with

a

computedsynthetic

reality

of

bits

rather

than

atoms.×

Helmet-mounted

displays

eventually

gave

way

to

less

alienating

display

technologies,such

as

the

recursively

named

C.A.V.E

(CAVE

automatic

virtual

environment)system,

which

projected

a

synthetic

environment

on

rear

projection

flats

and

motion-tracked

the

user

to

derive

and

update

his

or

her

position

in

the

simulated

space.×

頭部攜帶設(shè)計(jì)最終讓位于C.A.V.E.系統(tǒng)，該系統(tǒng)能追蹤人物運(yùn)動(dòng)，并通過(guò)背投的方式模擬環(huán)境變化。RELATED

WORK×

Infinity-by-Nine×

Bruce×

Infinity-by-Nine

is

in

part

inspired

by

results

obtained

in

“Bruce,”

an

unpublished2002

project

by

Surj

Patel

of

the

MIT

Media

Lab’s

Object-Based

Media

Group.

Agrid

of

networked,

color

controllable

LED

lights

was

placed

on

the

ceiling

above

alarge-screen

television.

Dynamic

“l(fā)ighting

tracks,”

creating

not

just

room

lighting

butalso

motion

effects,

were

authored

in

the

group’s

Isis

scripting

language

for

scenesfrom

several

films,

including

Blade

Runner

and

Clue.

Viewers

reported

that

theyperceived

an

increased

sense

of

involvement

with

the

films×

電視上面有LED燈軌道，隨著電影情節(jié)變化而變化×

Ambilight× It

does

not

collect

or

computeany

knowledge

of

the

sourcemedia

other

than

the

averagedcolor

information.×

平均色×

BBC

Surround

Video×

Microsoft’s

IllumiRoomRELATED

WORK×

Narratarium×

The

Aesthetiscope×

The

Aesthetiscope,

built

by

Patricia

Maes

and

Hugo

Liu,

creates

a

colorful

visualdisplay

based

upon

the

aesthetic

character

of

a

written

text

[Liu

et

al.

2005].

Thecolor

palette

is

determined

through

psycho-physiological

heuristics

that

apply

acustomized

Jungian

analytic

framework

to

the

text.

While

projected

on

a

full-scalewall

of

the

“Living

Room

of

the

Future,”

the

display

never

attempts

to

envelop

theuser

in

an

immersive

experience.

The

Aesthetiscope

could

be

an

excellent

additionto

the

modes

available

on

the

Narratarium

and

allow

users

to

explore

a

spatiallyimmersive

experience

of

the

text.×

通過(guò)文本分析產(chǎn)生色板，投影在起居室的墻上，帶給人們沉浸體驗(yàn)?！?/p>

Other

360-Degree

Projections

and

DisplaysRELATED

WORK×

3D

Telepresence

Chair×

Pepper’s

Ghost

佩珀?duì)柣孟蟆?/p>

DVE

Telepresence×

Holografika×

Teleconferencing

and

Multi-ViewSYSTEM

DEVELOPMENT×

Infinity-by-Nine×視頻SYSTEM

DEVELOPMENT×

The

NarratariumSYSTEM

DEVELOPMENT×

3D

Telepresence

ChairUSER

STUDIES×

Infinity-by-Nine× The

user

study

conducted

for

Infinity-by-Nine

took

place

in

a

simulated

living

room

environment

inthe

Object-Based

Media

Group

area

on

the

4th

floor

of

the

Media

Lab’s

classic

building.

The

testliving

room

included

three

small

couches

arrayed

before

a

large

screen

television

set

at

acomfortable

distance.

Two

projection

screens

were

hung

from

the

ceiling

to

simulate

the

sidewalls

of

the

room,

while

a

third

screen

was

suspended

above

the

space

to

operate

as

a

ceiling.Each

screen

was

slightly

angled

inward

to

create

a

foreshortening

or

small

“forced

perspective”effect

that

emphasizes

the

relationship

of

the

imagery

on

the

screen

to

its

origin

on

the

maintelevision.

The

side

and

top

projection

surfaces

were

luminance-

and

color-matched

as

closely

aspossible

to

the

main

video

screen

to

help

create

a

continuous

light

field.×

Entrance

Questionnaire× The

questions

on

the

Entrance

Questionnaire

were

designed

to

establish

the

home

televisionwatching

habits

of

the

participants.×

了解用戶觀影習(xí)慣×××Do

you

enjoy

watching

films

and/or

television?Both?喜好Do

you

enjoy

films

at

home

or

the

theater?Both?地點(diǎn)If

you

enjoy

watching

films

at

home,what

do

you

usually

watch

them

on?Circle

all

that

apply.終端×Television×Computer

Monitor×Projected×Hand

Held

Device

(Ipad,

Tablet,

Smart

Phone)×Other

(Please

Explain)×4.For

your

favorite

way

to

watch

film

or

television:距離還是尺寸？×A)

About

how

big

is

the

screen

or

projected

image

that

you

watch?×B)

How

close

or

far

do

you

usually

sit?×5.Do

you

enjoy

watching

stereoscopic(3D)films?If

so,what

do

you

like

about

the

experience?If

not,what

do

you

dislike

aboutit?3D體驗(yàn)×6.Do

you

watch

films

or

television

alone?If

not,with

whom

do

you

watch?和誰(shuí)？×

Exit

Questionnaire× The

Exit

Questionnaire

was

designed

to

allow

the

subjects

to

offer

qualitative

observations

abouttheir

experience

viewing

a

feature

film

on

Infinity-by-Nine.×

提供體驗(yàn)意見(jiàn)和建議×

1.What

did

you

think

about

the

experience?(Any

or

all

thoughts

are

appreciated)評(píng)價(jià)×

2.Did

you

find

the

peripheral

content

distracting?干擾×

3.If

you

have

watched

stereoscopic

(3D)films,did

you

find

this

experience

more

or

lessenjoyable

than

watching

a

stereoscopic

film?

比較×

4.Did

you

find

your

experience

with

Infinity-by-Nine

more

similar

to

watching

a

film

at

home,orin

the

theater?相近體驗(yàn)×

5.The

purpose

of

Infinity-by-Nine

is

to

create

a

sense

of“immersion,”or

emotional

and

cognitiveengagement,with

filmed

content.Did

you

feel

that

watching

a

film

with

the

peripheral

content

wasmore

or

less

immersive

or

engaging

than

watching

a

film

without

it?沉浸感評(píng)價(jià)×

6.What

other

films

do

you

think

would

work

well

on

Infinity-by-Nine?電影選擇×

7.What

other

forms

of

entertainment

do

you

think

would

work

well

onInfinity-by-Nine?形式選擇USER

TESTING

CONCLUSIONS×

The

Entrance

Questionnaire

indicated

that

most

of

the

subjects

prefer

to

watch

filmswith

family

or

friends,and

would

rather

watch

films

in

a

theater

than

at

home.

傾向于與親友一起看，傾向于在電影院看×

More

than

80

percent

of

the

subjects

indicated

that

Infinity-by-Nine

increased

theirsense

of

immersion.

絕大多數(shù)人表示增加了沉浸感× A

surprising

result

was

that

several

subjects

found

that

this

immersive

qualityhelped

them

bring

their

attention

back

to

the

film

if

it

wandered.甚至有人表示幫助集中注意力×

However,the

criticism

that

stood

out

most

clearly

was

that

many

of

the

subjectswere

aware

of

moments

when

the

optical

flow

tracking

failed

to

find

enough

salientfeatures

to

derive

a

global

camera

solve.一些無(wú)關(guān)緊要的不足，光刺激略強(qiáng)等等。USER

STUDIES×

The

Narratarium×

In

its

current

configuration,

the

Narratarium

is

fully

programmed

to

respond

to

Underthe

Same

Moon,

or

to

respond

to

extemporaneous

speech

with

a

limited

palette

oftext,

color,

and

shape.

The

current

modalities

provide

a

fundamental

introduction

tothe

possibilities

of

the

system.

However,

at

this

time

we

feel

that

more

modalities,including

a

deeper

integration

with

ConceptNet5,

need

to

be

added

to

the

Narratarium

in

order

to

properly

perform

a

user

study.×

當(dāng)前的模式只是提供了一種解決方案的可能性，比如對(duì)一本書(shū)，或者一段即興演講，產(chǎn)生有限的色板，要開(kāi)展用戶研究，還需要更深層次的整合。USER

STUDIES×

3D

Telepresence

Chair×An

early

version

of

the

Telepresence

Chair

was

used

to

allow

Media

Lab

Director

Joichi

Ito

to

appear

as

a“hologram”

to

the

Board

of

Directors

of

Singtel

at

their

annual

meeting.

The

board

was

in

residence

on

the6th

floor

of

the

MIT

Media

Lab

in

Cambridge,

Massachusetts,

while

Mr.

Ito

was

in

Phoenix,

Arizona.

TheChair

utilized

Skype

in

full

screen

mode

and

was

limited

to

two

dimensions

for

this

early

iteration.

Two-waycommunication

went

well,

even

considering

that

no

specialized

teleconferencing

hardware

or

dedicated

channels

were

being

used.

Mr.

Ito

was

utilizing

a

wireless

connection

provided

by

his

hotel

and

theTelepresence

Chair

was

connected

to

MIT’s

standard

wired

network.

The

members

of

Singtel’s

board

weredelighted

with

the

novelty

of

the

approach

and

enjoyed

communicating

with

Mr.

Ito

as

if

he

were

in

theroom.×In

order

to

amplify

Mr.

Ito’s

voice

so

that

it

could

be

heard

by

a

large

group,

his

audio

feed

was

input

intothe

room’