Introduction
Hi, I'm Šárka Sochorová, and together with my Co-Founder Ondřej Jamriška we started Secret Weapons, a software company that makes powerful tools for visual artists. We met at the Czech Technical University in Prague, where Ondřej worked as a computer graphics researcher, and I joined their team to do my Ph.D. I had a background in Computer Science and Traditional Animation, so Ondřej's research was fascinating to me because it was the perfect merge of art and technology. He co-authored several papers on example-based synthesis (LazyFluids, StyLit, FaceStyle), which is a style-transfer method where you provide an example of the look you want to achieve, and the algorithm applies it on a 3D model, CG simulation, or photograph.
介绍
嗨，我是Šárka Sochorová，我和我的联合创始人Ondřej Jamriška一起创办了Secret Weapons，这是一家为视觉艺术家制作强大工具的软件公司。我们在布拉格的捷克技术大学相遇，Ondřej在那里担任计算机图形学研究员，我加入了他们的团队攻读博士学位。我有计算机科学和传统动画的背景，所以Ondřej的研究对我来说很吸引人，因为它是艺术和技术的完美结合。他与人合著了几篇关于基于示例的合成的论文（LazyFluids，StyLit，FaceStyle），这是一种样式转移方法，您可以在其中提供要实现的外观的示例，并且该算法将其应用于3D模型，CG模拟或照片。



Soon after I joined, we started experimenting with transferring the style from paintings to arbitrary videos. That was the birth of EbSynth and the first project we worked on together. Recently, we published our latest research on practical pigment mixing for digital painting. Surprisingly, the majority of painting programs don’t mix colors as real paints even though the mathematical model for simulating it is well known. To make realistic pigment mixing more practical, we created Mixbox. It hides all the physics inside an RGB-in RGB-out box, making it easy to implement into any painting software.
在我加入后不久，我们开始尝试将这种风格从绘画转移到任意视频中。这是EbSynth的诞生，也是我们共同合作的第一个项目。最近，我们发表了关于数字绘画中实用颜料混合的最新研究。令人惊讶的是，大多数绘画程序不会将颜色混合为真正的油漆，即使模拟它的数学模型是众所周知的。为了使逼真的颜料混合更实用，我们创建了Mixbox。它将所有物理场隐藏在RGB输入RGB输出框中，使其易于实现到任何绘画软件中。



The Origins of Secret Weapons and EbSynth
Doing graphics research is really exciting. But after years of doing SIGGRAPH papers, Ondřej really wanted to start getting those techniques into the hands of artists. I loved the idea because I never saw myself as an academic anyway. We both enjoyed doing research, but we really wanted to turn it into something practical that regular people could actually use. We considered different options and decided that starting our own company is the best way to achieve this. Among EbSynth, we had a list of other really cool tools that we wanted to make. We thought that these tools could be the secret weapons of animators and VFX artists. That's why we chose the name ‘Secret Weapons’.
秘密武器和EbSynth的起源
做图形研究真的很令人兴奋。但是，经过多年的SIGGRAPH论文，Ondřej真的想开始将这些技术交到艺术家手中。我喜欢这个想法，因为无论如何，我从未将自己视为学者。我们都喜欢做研究，但我们真的想把它变成普通人可以实际使用的东西。我们考虑了不同的选择，并决定创办自己的公司是实现这一目标的最佳方式。在EbSynth中，我们列出了我们想要制作的其他非常酷的工具。我们认为这些工具可能是动画师和视觉特效艺术家的秘密武器。这就是为什么我们选择了“秘密武器”这个名字。

The original inspiration for EbSynth came from Jakub Javora. He is an amazing concept artist and he kept looking for ways to bring his paintings to life with some movement. So, Ondřej made him a tool to stick a painting onto a video and make it move. It was kind of primitive and the results were crude, but the idea was there. A few years later we were approached by a production company that looked for help with a rotoscoped movie they were trying to get off the ground. This made us revive the tool and enhance it with some synthesis technology we worked on in the meantime. It kept getting better and better, and at some point, we wanted to share it with the world.
EbSynth最初的灵感来自Jakub Javora。他是一位了不起的概念艺术家，他一直在寻找通过一些运动使他的画作栩栩如生的方法。因此，Ondřej为他制作了一个工具，可以将一幅画粘贴到视频上并使其移动。它有点原始，结果是粗糙的，但这个想法就在那里。几年后，一家制片公司找到我们，他们试图通过一部他们试图启动的转描电影寻求帮助。这使我们恢复了该工具，并通过我们在此期间开发的一些合成技术对其进行了增强。它变得越来越好，在某个时候，我们想与世界分享它。

更多精彩图片 请登录

Here's a nice throwback to the SIGGRAPH computer graphics conference, where we presented our EbSynth technical paper. From the left: Jakub, Šárka, Ondřej:

这是SIGGRAPH计算机图形学会议的一个很好的回顾，我们在那里展示了我们的EbSynth技术论文。左起：雅各布、萨尔卡、翁德热伊:

更多精彩图片 请登录

The Tech Behind It
Although many people think there is AI involved, there really isn't. The tech behind EbSynth is 'Example-Based Synthesis', that's where the name EbSynth comes from. You provide a video and a painted keyframe – an example of your style. EbSynth breaks your painting into many tiny pieces, like a jigsaw puzzle. It then uses those pieces to assemble (synthesize) all the remaining video frames.
背后的技术
虽然许多人认为人工智能参与其中，但实际上并没有。EbSynth背后的技术是“基于示例的合成”，这就是EbSynth这个名字的来源。您提供了一个视频和一个绘制的关键帧 - 您的样式的示例。EbSynth将你的画分解成许多小块，就像一个拼图游戏。然后，它使用这些片段来组装（合成）所有剩余的视频帧。

更多精彩图片 请登录

That way, EbSynth always works only with what the artist paints. It never introduces anything new that wasn't shown in the painted keyframe. This means that the artist actually has control over the stylized output. By modifying even small details in the keyframe painting, they can alter the visual style of the final animation.
这样，EbSynth总是只与艺术家画的东西一起工作。它从不引入任何未在绘制的关键帧中显示的新内容。这意味着艺术家实际上可以控制风格化的输出。通过修改关键帧绘制中的小细节，他们可以改变最终动画的视觉风格。

How It Works
EbSynth should be pretty easy to use, but there are a few things to understand if you want really nice results. Let me summarize the workflow. First, you need a video. It can be live-action footage, a hand-drawn animation, or a 3D render. You then pick one keyframe and paint over it or edit it as you like. The video and the painted keyframe are your inputs. You feed them to EbSynth, and it distributes the style of your keyframe to all the remaining frames.
它是如何运作的
EbSynth应该非常易于使用，但是如果您想要非常好的结果，则需要了解一些事情。让我总结一下工作流程。首先，你需要一个视频。它可以是实景镜头、手绘动画或 3D 渲染。然后，选择一个关键帧并在上面绘制或根据需要对其进行编辑。视频和绘制的关键帧是您的输入。将它们提供给 EbSynth，它会将关键帧的样式分配给所有剩余的帧。

更多精彩图片 请登录

Of course, it's almost never this easy, so let's get into more details.  

a) Keyframe choice
The keyframe should reveal as much of the scene as possible because when you paint over it, you give EbSynth examples of how things should look. So, suppose I have a shot where a person suddenly smiles. In that case, the keyframe should contain that person's open eyes and an open mouth with teeth if they are visible at any point. You show EbSynth how you want the eyes and teeth to look. With that information, it can create both: the frames where the eyes and mouth are closed and when they are open.
当然，这几乎从未如此简单，所以让我们进入更多细节。

a） 关键帧选择
关键帧应尽可能多地显示场景，因为当您在上面绘画时，您会给出 EbSynth 示例，说明事物的外观。所以，假设我有一个镜头，一个人突然微笑。在这种情况下，关键帧应包含该人睁开的眼睛和带有牙齿的张开嘴巴（如果它们在任何时候都可见）。你向EbSynth展示你想要的眼睛和牙齿的样子。有了这些信息，它可以创建两者：眼睛和嘴巴闭合的框架以及眼睛和嘴巴打开的框架。

b) Match the painting with the video frame
Make sure that your painting matches the video frame as closely as possible. If a person on your video isn't wearing a hat and you paint it in the keyframe, you'll get some ugly artifacts. The animation is guided by the video. If you want a person with a hat in your painted result, make the actor wear one in the video. Use props – it's fun!
b）将绘画与视频帧
匹配 确保您的绘画尽可能与视频帧匹配。如果视频中的某个人没有戴帽子，而你在关键帧中画了它，你会得到一些丑陋的伪像。动画由视频引导。如果你想在画结果中画一个戴着帽子的人，让演员在视频中戴一顶帽子。使用道具 - 这很有趣！

Jakub Javora’s project:
雅各布·雅沃拉的项目:

更多精彩图片 请登录

更多精彩图片 请登录

c) More Keyframes
Sometimes, one keyframe is enough. But you usually need more, and they need to be consistent. To achieve that, always start with just one keyframe. Feed it to EbSynth and get your first animated sequence out. Usually, the frames close to the keyframe are fine, but the further you get, the more broken they look. That's a good location for a new keyframe but don't paint it from scratch. Use the broken result of the first EbSynth run and fix it. Sometimes it needs just a couple of brushstrokes.
c） 更多关键帧
有时，一个关键帧就足够了。但你通常需要更多，而且它们需要保持一致。要实现这一点，请始终仅从一个关键帧开始。将其提供给EbSynth，并获取您的第一个动画序列。通常，靠近关键帧的帧很好，但是越远，它们看起来就越破损。对于新关键帧来说，这是一个不错的位置，但不要从头开始绘制它。使用第一次 EbSynth 运行的损坏结果并进行修复。有时它只需要几个笔触。

Creating new keyframes by fixing the broken frames from the previous synthesis runs makes sure that your keyframes are consistent. This is super important for further blending.

d) Blending multiple keyframes
Once you have a new keyframe, feed it to EbSynth and alter the output sequence frame range. Let me give you an example. Suppose my video sequence runs from frames 0 to 100, and my first keyframe is frame number 50. I would run the first synthesis over the whole video – from 0 to 100. Let’s say that the result gets really messy at frame number 25. So, I fix it with a couple of brushstrokes and turn it into my second keyframe. This time, I only run the synthesis from frames 0 to 50. My third keyframe is frame number 75, so I run the last synthesis from 50 to 100. Now, I have three stylized image sequences. All I need to do to get the final result is blend them together.
通过修复先前合成运行中断开的帧来创建新的关键帧可确保关键帧一致。这对于进一步混合非常重要。

d） 混合多个关键帧
有了新的关键帧后，将其提供给 EbSynth 并更改输出序列帧范围。让我举个例子。假设我的视频序列从第 0 帧运行到第 100 帧，我的第一个关键帧是帧编号 50。我会对整个视频进行第一次合成 - 从0到100。假设结果在第 25 帧变得非常混乱。所以，我用几个笔触修复了它，并将其转换为我的第二个关键帧。这一次，我只运行从0帧到50帧的合成。我的第三个关键帧是帧编号 75，因此我运行了从 50 到 100 的最后一次合成。现在，我有三个风格化的图像序列。为了获得最终结果，我需要做的就是将它们混合在一起。

更多精彩图片 请登录

EbSynth does not perform the blending by itself, but it can export your project to After Effects and prepare the initial cross-fades for you. You can then edit them if you want or just render the final video. If you don't have After Effects, just use another software to perform simple cross-fading.

e) Use layers
The last and maybe the most important tip is to work with layers. EbSynth can handle the alpha channel, and it will give you results on a transparent background. That way, you can separate your foreground figures from the background and work on them individually.
EbSynth 本身不会执行混合，但它可以将项目导出到 After Effects 并为您准备初始交叉淡入淡出。然后，您可以根据需要编辑它们，或者只渲染最终视频。如果您没有 After Effects，只需使用其他软件即可执行简单的交叉淡入淡出。

e）使用图层
最后一个，也许是最重要的提示是使用图层。EbSynth可以处理Alpha通道，它将在透明背景上为您提供结果。这样，您就可以将前景人物与背景分开，并单独处理它们。

更多精彩图片 请登录

This will immediately give you cleaner edges in your result. By dividing your scene into more layers, you get more control over details. For example, you can have one keyframe for the body and three keyframes for the head, if needed. Just paint the head on a transparent background and feed it to EbSynth. It will give you an animated face on a transparent background that you can later compose with the rest of the body. Using layers leads to a bigger compositing and blending project, but you can achieve wonderful results.
这将立即在结果中为您提供更清晰的边缘。通过将场景划分为更多图层，您可以更好地控制细节。例如，如果需要，可以有一个关键帧用于正文，三个关键帧用于头部。只需在透明背景上绘制头部并将其提供给EbSynth即可。它会在透明背景上给你一个动画的脸，你以后可以与身体的其余部分一起组成。使用图层会导致更大的合成和混合项目，但您可以获得出色的效果。

Changes and Challenges
During the development, there were many technical challenges: detail preservation, temporal coherence, speed, etc. But these were expected issues, and especially Ondřej was really great at solving them. However, neither of us has any experience in marketing. So, after releasing the Alpha version, it was pretty challenging to let people know EbSynth exists. We wanted to get it to artists and see if they find it useful or not. Based on the reaction, we were either going to keep working on it or drop it. That's why we are so proud of all the artists who embraced EbSynth and included it in their workflows. There are so many exciting and wonderful animations made with EbSynth on the internet today. That makes us truly happy. It motivates us to keep going and gives us the reason to finish EbSynth 1.0.
变化与挑战
在开发过程中，存在许多技术挑战：细节保留，时间一致性，速度等。但这些都是预料之中的问题，特别是Ondřej非常擅长解决这些问题。但是，我们俩都没有任何营销经验。因此，在发布Alpha版本后，让人们知道EbSynth的存在是非常具有挑战性的。我们想把它交给艺术家，看看他们是否觉得有用。根据反应，我们要么继续努力，要么放弃它。这就是为什么我们为所有接受EbSynth并将其纳入工作流程的艺术家感到自豪的原因。今天在互联网上有很多令人兴奋和精彩的动画使用EbSynth制作。这让我们真正感到高兴。它激励我们继续前进，并给了我们完成EbSynth 1.0的理由。

Joel Haver is particularly special to us because he made EbSynth’s animation a part of his handwriting and that’s just amazing, he’s a legend.

We haven’t introduced that many changes yet. The Alpha version was the first prototype, just to see if people find it interesting and see some value in it or not. When we noticed that artists started using it, we released an improved Beta version. It is a lot faster, the interface is a bit more user-friendly, and we added direct export to After Effects to make the blending easier. The most significant change will come with the release of EbSynth 1.0, which will have a completely new and interactive user interface to make the workflow smooth and efficient.
Joel Haver对我们来说特别特别，因为他让EbSynth的动画成为他笔迹的一部分，这真是太神奇了，他是一个传奇。

我们还没有引入那么多的更改。Alpha版本是第一个原型，只是为了看看人们是否觉得它很有趣，并看到它是否有一些价值。当我们注意到艺术家开始使用它时，我们发布了一个改进的Beta版本。它的速度要快得多，界面更加用户友好，并且我们在After Effects中添加了直接导出功能，以使混合更容易。最重大的变化将来自EbSynth 1.0的发布，它将具有全新的交互式用户界面，使工作流程顺畅高效。

Competition
We are definitely keeping a close eye on similar tools. Rather than being scared that EbSynth might get replaced, we’re focusing on making a useful and fun-to-use tool. We believe that if we do it right, we can earn a spot in the industry. The scenario has been pretty clear to us from the beginning. We want to make EbSynth available to everyone: individual artists, small studios, and also big studios, who usually have specific workflows and need some customization. So, we knew that we didn’t want to sell just the technology. We wanted to finish the product ourselves and offer it to the users. And that’s still our goal.
竞争
我们肯定会密切关注类似的工具。我们并不担心EbSynth可能会被取代，而是专注于制作一个有用且易于使用的工具。我们相信，如果我们做得对，我们可以在行业中赢得一席之地。从一开始，我们就很清楚这种情况。我们希望让每个人都可以使用EbSynth：个人艺术家，小型工作室以及大型工作室，他们通常具有特定的工作流程并且需要一些自定义。所以，我们知道我们不想只出售技术。我们希望自己完成产品并将其提供给用户。这仍然是我们的目标。

Plans
Our goal is to make EbSynth interactive and easy to use. So we're currently focusing on speed and user interface. The main challenge is to improve the performance to a point that enables real-time preview. This will allow artists to see the effects of their changes immediately without having to wait for the offline render to finish. Besides preview, the interface will also contain an interactive timeline to make the process of adding, editing, and blending keyframes fast and smooth. Essentially, we want artists to focus purely on the creative side of the process and not get slowed down by complicated workflows. We're aiming for the release in Q3 2022.
计划
我们的目标是使EbSynth具有交互性和易用性。因此，我们目前专注于速度和用户界面。主要挑战是将性能提高到能够实现实时预览的程度。这将允许美术师立即看到其更改的效果，而无需等待脱机渲染完成。除了预览之外，该界面还将包含一个交互式时间轴，以使添加，编辑和混合关键帧的过程快速流畅。从本质上讲，我们希望艺术家完全专注于流程的创意方面，而不是因复杂的工作流程而减慢速度。我们的目标是在2022年第三季度发布。

The Future of Generative Art
It looks like there is a bit of a bubble around AI today. Anything with the AI or ML label seems sensational, including generative art. So it probably has some future, because people like to discuss it, researchers want to improve it, and artists like to play with it. But I'm not concerned that generative art would actually replace real artists. Aaron Hertzmann wrote a very nice, in-depth essay “Computers do not Make Art, People Do” on this topic.
生成艺术的未来
看起来今天围绕人工智能有一点泡沫。任何带有AI或ML标签的东西似乎都是耸人听闻的，包括生成艺术。因此，它可能有一些未来，因为人们喜欢讨论它，研究人员想要改进它，艺术家喜欢玩它。但我并不担心生成艺术实际上会取代真正的艺术家。Aaron Hertzmann就这个话题写了一篇非常漂亮，深入的文章“计算机不创造艺术，人们会做”。

We noticed style transfer tools based on AI such as Artbreeder or Toonify!
我们注意到基于AI的样式转移工具，例如Artbreeder或Toonify！

Toonify – sometimes it works, sometimes it doesn’t. Take it or leave it.:
Toonify - 有时它有效，有时它不起作用。要就要不要拉倒。:

更多精彩图片 请登录

Currently, the results from tools like these are somewhat unpredictable and hard to control, which limits their utility for production use. However, these tools can be a good source of fun and inspiration, which is positive.
目前，这些工具的结果有些不可预测且难以控制，这限制了它们在生产中的实用性。但是，这些工具可以成为乐趣和灵感的良好来源，这是积极的。

Introduction
Hi, I'm Šárka Sochorová, and together with my Co-Founder Ondřej Jamriška we started Secret Weapons, a software company that makes powerful tools for visual artists. We met at the Czech Technical University in Prague, where Ondřej worked as a computer graphics researcher, and I joined their team to do my Ph.D. I had a background in Computer Science and Traditional Animation, so Ondřej's research was fascinating to me because it was the perfect merge of art and technology. He co-authored several papers on example-based synthesis (LazyFluids, StyLit, FaceStyle), which is a style-transfer method where you provide an example of the look you want to achieve, and the algorithm applies it on a 3D model, CG simulation, or photograph.
介绍
嗨，我是Šárka Sochorová，我和我的联合创始人Ondřej Jamriška一起创办了Secret Weapons，这是一家为视觉艺术家制作强大工具的软件公司。我们在布拉格的捷克技术大学相遇，Ondřej在那里担任计算机图形学研究员，我加入了他们的团队攻读博士学位。我有计算机科学和传统动画的背景，所以Ondřej的研究对我来说很吸引人，因为它是艺术和技术的完美结合。他与人合著了几篇关于基于示例的合成的论文（LazyFluids，StyLit，FaceStyle），这是一种样式转移方法，您可以在其中提供要实现的外观的示例，并且该算法将其应用于3D模型，CG模拟或照片。



Soon after I joined, we started experimenting with transferring the style from paintings to arbitrary videos. That was the birth of EbSynth and the first project we worked on together. Recently, we published our latest research on practical pigment mixing for digital painting. Surprisingly, the majority of painting programs don’t mix colors as real paints even though the mathematical model for simulating it is well known. To make realistic pigment mixing more practical, we created Mixbox. It hides all the physics inside an RGB-in RGB-out box, making it easy to implement into any painting software.
在我加入后不久，我们开始尝试将这种风格从绘画转移到任意视频中。这是EbSynth的诞生，也是我们共同合作的第一个项目。最近，我们发表了关于数字绘画中实用颜料混合的最新研究。令人惊讶的是，大多数绘画程序不会将颜色混合为真正的油漆，即使模拟它的数学模型是众所周知的。为了使逼真的颜料混合更实用，我们创建了Mixbox。它将所有物理场隐藏在RGB输入RGB输出框中，使其易于实现到任何绘画软件中。



The Origins of Secret Weapons and EbSynth
Doing graphics research is really exciting. But after years of doing SIGGRAPH papers, Ondřej really wanted to start getting those techniques into the hands of artists. I loved the idea because I never saw myself as an academic anyway. We both enjoyed doing research, but we really wanted to turn it into something practical that regular people could actually use. We considered different options and decided that starting our own company is the best way to achieve this. Among EbSynth, we had a list of other really cool tools that we wanted to make. We thought that these tools could be the secret weapons of animators and VFX artists. That's why we chose the name ‘Secret Weapons’.
秘密武器和EbSynth的起源
做图形研究真的很令人兴奋。但是，经过多年的SIGGRAPH论文，Ondřej真的想开始将这些技术交到艺术家手中。我喜欢这个想法，因为无论如何，我从未将自己视为学者。我们都喜欢做研究，但我们真的想把它变成普通人可以实际使用的东西。我们考虑了不同的选择，并决定创办自己的公司是实现这一目标的最佳方式。在EbSynth中，我们列出了我们想要制作的其他非常酷的工具。我们认为这些工具可能是动画师和视觉特效艺术家的秘密武器。这就是为什么我们选择了“秘密武器”这个名字。

The original inspiration for EbSynth came from Jakub Javora. He is an amazing concept artist and he kept looking for ways to bring his paintings to life with some movement. So, Ondřej made him a tool to stick a painting onto a video and make it move. It was kind of primitive and the results were crude, but the idea was there. A few years later we were approached by a production company that looked for help with a rotoscoped movie they were trying to get off the ground. This made us revive the tool and enhance it with some synthesis technology we worked on in the meantime. It kept getting better and better, and at some point, we wanted to share it with the world.
EbSynth最初的灵感来自Jakub Javora。他是一位了不起的概念艺术家，他一直在寻找通过一些运动使他的画作栩栩如生的方法。因此，Ondřej为他制作了一个工具，可以将一幅画粘贴到视频上并使其移动。它有点原始，结果是粗糙的，但这个想法就在那里。几年后，一家制片公司找到我们，他们试图通过一部他们试图启动的转描电影寻求帮助。这使我们恢复了该工具，并通过我们在此期间开发的一些合成技术对其进行了增强。它变得越来越好，在某个时候，我们想与世界分享它。

更多精彩图片 请登录

Here's a nice throwback to the SIGGRAPH computer graphics conference, where we presented our EbSynth technical paper. From the left: Jakub, Šárka, Ondřej:

这是SIGGRAPH计算机图形学会议的一个很好的回顾，我们在那里展示了我们的EbSynth技术论文。左起：雅各布、萨尔卡、翁德热伊:

更多精彩图片 请登录

The Tech Behind It
Although many people think there is AI involved, there really isn't. The tech behind EbSynth is 'Example-Based Synthesis', that's where the name EbSynth comes from. You provide a video and a painted keyframe – an example of your style. EbSynth breaks your painting into many tiny pieces, like a jigsaw puzzle. It then uses those pieces to assemble (synthesize) all the remaining video frames.
背后的技术
虽然许多人认为人工智能参与其中，但实际上并没有。EbSynth背后的技术是“基于示例的合成”，这就是EbSynth这个名字的来源。您提供了一个视频和一个绘制的关键帧 - 您的样式的示例。EbSynth将你的画分解成许多小块，就像一个拼图游戏。然后，它使用这些片段来组装（合成）所有剩余的视频帧。

更多精彩图片 请登录

That way, EbSynth always works only with what the artist paints. It never introduces anything new that wasn't shown in the painted keyframe. This means that the artist actually has control over the stylized output. By modifying even small details in the keyframe painting, they can alter the visual style of the final animation.
这样，EbSynth总是只与艺术家画的东西一起工作。它从不引入任何未在绘制的关键帧中显示的新内容。这意味着艺术家实际上可以控制风格化的输出。通过修改关键帧绘制中的小细节，他们可以改变最终动画的视觉风格。

How It Works
EbSynth should be pretty easy to use, but there are a few things to understand if you want really nice results. Let me summarize the workflow. First, you need a video. It can be live-action footage, a hand-drawn animation, or a 3D render. You then pick one keyframe and paint over it or edit it as you like. The video and the painted keyframe are your inputs. You feed them to EbSynth, and it distributes the style of your keyframe to all the remaining frames.
它是如何运作的
EbSynth应该非常易于使用，但是如果您想要非常好的结果，则需要了解一些事情。让我总结一下工作流程。首先，你需要一个视频。它可以是实景镜头、手绘动画或 3D 渲染。然后，选择一个关键帧并在上面绘制或根据需要对其进行编辑。视频和绘制的关键帧是您的输入。将它们提供给 EbSynth，它会将关键帧的样式分配给所有剩余的帧。

更多精彩图片 请登录

Of course, it's almost never this easy, so let's get into more details.  

a) Keyframe choice
The keyframe should reveal as much of the scene as possible because when you paint over it, you give EbSynth examples of how things should look. So, suppose I have a shot where a person suddenly smiles. In that case, the keyframe should contain that person's open eyes and an open mouth with teeth if they are visible at any point. You show EbSynth how you want the eyes and teeth to look. With that information, it can create both: the frames where the eyes and mouth are closed and when they are open.
当然，这几乎从未如此简单，所以让我们进入更多细节。

a） 关键帧选择
关键帧应尽可能多地显示场景，因为当您在上面绘画时，您会给出 EbSynth 示例，说明事物的外观。所以，假设我有一个镜头，一个人突然微笑。在这种情况下，关键帧应包含该人睁开的眼睛和带有牙齿的张开嘴巴（如果它们在任何时候都可见）。你向EbSynth展示你想要的眼睛和牙齿的样子。有了这些信息，它可以创建两者：眼睛和嘴巴闭合的框架以及眼睛和嘴巴打开的框架。

b) Match the painting with the video frame
Make sure that your painting matches the video frame as closely as possible. If a person on your video isn't wearing a hat and you paint it in the keyframe, you'll get some ugly artifacts. The animation is guided by the video. If you want a person with a hat in your painted result, make the actor wear one in the video. Use props – it's fun!
b）将绘画与视频帧
匹配 确保您的绘画尽可能与视频帧匹配。如果视频中的某个人没有戴帽子，而你在关键帧中画了它，你会得到一些丑陋的伪像。动画由视频引导。如果你想在画结果中画一个戴着帽子的人，让演员在视频中戴一顶帽子。使用道具 - 这很有趣！

Jakub Javora’s project:
雅各布·雅沃拉的项目:

更多精彩图片 请登录

更多精彩图片 请登录

c) More Keyframes
Sometimes, one keyframe is enough. But you usually need more, and they need to be consistent. To achieve that, always start with just one keyframe. Feed it to EbSynth and get your first animated sequence out. Usually, the frames close to the keyframe are fine, but the further you get, the more broken they look. That's a good location for a new keyframe but don't paint it from scratch. Use the broken result of the first EbSynth run and fix it. Sometimes it needs just a couple of brushstrokes.
c） 更多关键帧
有时，一个关键帧就足够了。但你通常需要更多，而且它们需要保持一致。要实现这一点，请始终仅从一个关键帧开始。将其提供给EbSynth，并获取您的第一个动画序列。通常，靠近关键帧的帧很好，但是越远，它们看起来就越破损。对于新关键帧来说，这是一个不错的位置，但不要从头开始绘制它。使用第一次 EbSynth 运行的损坏结果并进行修复。有时它只需要几个笔触。

Creating new keyframes by fixing the broken frames from the previous synthesis runs makes sure that your keyframes are consistent. This is super important for further blending.

d) Blending multiple keyframes
Once you have a new keyframe, feed it to EbSynth and alter the output sequence frame range. Let me give you an example. Suppose my video sequence runs from frames 0 to 100, and my first keyframe is frame number 50. I would run the first synthesis over the whole video – from 0 to 100. Let’s say that the result gets really messy at frame number 25. So, I fix it with a couple of brushstrokes and turn it into my second keyframe. This time, I only run the synthesis from frames 0 to 50. My third keyframe is frame number 75, so I run the last synthesis from 50 to 100. Now, I have three stylized image sequences. All I need to do to get the final result is blend them together.
通过修复先前合成运行中断开的帧来创建新的关键帧可确保关键帧一致。这对于进一步混合非常重要。

d） 混合多个关键帧
有了新的关键帧后，将其提供给 EbSynth 并更改输出序列帧范围。让我举个例子。假设我的视频序列从第 0 帧运行到第 100 帧，我的第一个关键帧是帧编号 50。我会对整个视频进行第一次合成 - 从0到100。假设结果在第 25 帧变得非常混乱。所以，我用几个笔触修复了它，并将其转换为我的第二个关键帧。这一次，我只运行从0帧到50帧的合成。我的第三个关键帧是帧编号 75，因此我运行了从 50 到 100 的最后一次合成。现在，我有三个风格化的图像序列。为了获得最终结果，我需要做的就是将它们混合在一起。

更多精彩图片 请登录

EbSynth does not perform the blending by itself, but it can export your project to After Effects and prepare the initial cross-fades for you. You can then edit them if you want or just render the final video. If you don't have After Effects, just use another software to perform simple cross-fading.

e) Use layers
The last and maybe the most important tip is to work with layers. EbSynth can handle the alpha channel, and it will give you results on a transparent background. That way, you can separate your foreground figures from the background and work on them individually.
EbSynth 本身不会执行混合，但它可以将项目导出到 After Effects 并为您准备初始交叉淡入淡出。然后，您可以根据需要编辑它们，或者只渲染最终视频。如果您没有 After Effects，只需使用其他软件即可执行简单的交叉淡入淡出。

e）使用图层
最后一个，也许是最重要的提示是使用图层。EbSynth可以处理Alpha通道，它将在透明背景上为您提供结果。这样，您就可以将前景人物与背景分开，并单独处理它们。

更多精彩图片 请登录

This will immediately give you cleaner edges in your result. By dividing your scene into more layers, you get more control over details. For example, you can have one keyframe for the body and three keyframes for the head, if needed. Just paint the head on a transparent background and feed it to EbSynth. It will give you an animated face on a transparent background that you can later compose with the rest of the body. Using layers leads to a bigger compositing and blending project, but you can achieve wonderful results.
这将立即在结果中为您提供更清晰的边缘。通过将场景划分为更多图层，您可以更好地控制细节。例如，如果需要，可以有一个关键帧用于正文，三个关键帧用于头部。只需在透明背景上绘制头部并将其提供给EbSynth即可。它会在透明背景上给你一个动画的脸，你以后可以与身体的其余部分一起组成。使用图层会导致更大的合成和混合项目，但您可以获得出色的效果。

Changes and Challenges
During the development, there were many technical challenges: detail preservation, temporal coherence, speed, etc. But these were expected issues, and especially Ondřej was really great at solving them. However, neither of us has any experience in marketing. So, after releasing the Alpha version, it was pretty challenging to let people know EbSynth exists. We wanted to get it to artists and see if they find it useful or not. Based on the reaction, we were either going to keep working on it or drop it. That's why we are so proud of all the artists who embraced EbSynth and included it in their workflows. There are so many exciting and wonderful animations made with EbSynth on the internet today. That makes us truly happy. It motivates us to keep going and gives us the reason to finish EbSynth 1.0.
变化与挑战
在开发过程中，存在许多技术挑战：细节保留，时间一致性，速度等。但这些都是预料之中的问题，特别是Ondřej非常擅长解决这些问题。但是，我们俩都没有任何营销经验。因此，在发布Alpha版本后，让人们知道EbSynth的存在是非常具有挑战性的。我们想把它交给艺术家，看看他们是否觉得有用。根据反应，我们要么继续努力，要么放弃它。这就是为什么我们为所有接受EbSynth并将其纳入工作流程的艺术家感到自豪的原因。今天在互联网上有很多令人兴奋和精彩的动画使用EbSynth制作。这让我们真正感到高兴。它激励我们继续前进，并给了我们完成EbSynth 1.0的理由。

Joel Haver is particularly special to us because he made EbSynth’s animation a part of his handwriting and that’s just amazing, he’s a legend.

We haven’t introduced that many changes yet. The Alpha version was the first prototype, just to see if people find it interesting and see some value in it or not. When we noticed that artists started using it, we released an improved Beta version. It is a lot faster, the interface is a bit more user-friendly, and we added direct export to After Effects to make the blending easier. The most significant change will come with the release of EbSynth 1.0, which will have a completely new and interactive user interface to make the workflow smooth and efficient.
Joel Haver对我们来说特别特别，因为他让EbSynth的动画成为他笔迹的一部分，这真是太神奇了，他是一个传奇。

我们还没有引入那么多的更改。Alpha版本是第一个原型，只是为了看看人们是否觉得它很有趣，并看到它是否有一些价值。当我们注意到艺术家开始使用它时，我们发布了一个改进的Beta版本。它的速度要快得多，界面更加用户友好，并且我们在After Effects中添加了直接导出功能，以使混合更容易。最重大的变化将来自EbSynth 1.0的发布，它将具有全新的交互式用户界面，使工作流程顺畅高效。

Competition
We are definitely keeping a close eye on similar tools. Rather than being scared that EbSynth might get replaced, we’re focusing on making a useful and fun-to-use tool. We believe that if we do it right, we can earn a spot in the industry. The scenario has been pretty clear to us from the beginning. We want to make EbSynth available to everyone: individual artists, small studios, and also big studios, who usually have specific workflows and need some customization. So, we knew that we didn’t want to sell just the technology. We wanted to finish the product ourselves and offer it to the users. And that’s still our goal.
竞争
我们肯定会密切关注类似的工具。我们并不担心EbSynth可能会被取代，而是专注于制作一个有用且易于使用的工具。我们相信，如果我们做得对，我们可以在行业中赢得一席之地。从一开始，我们就很清楚这种情况。我们希望让每个人都可以使用EbSynth：个人艺术家，小型工作室以及大型工作室，他们通常具有特定的工作流程并且需要一些自定义。所以，我们知道我们不想只出售技术。我们希望自己完成产品并将其提供给用户。这仍然是我们的目标。

Plans
Our goal is to make EbSynth interactive and easy to use. So we're currently focusing on speed and user interface. The main challenge is to improve the performance to a point that enables real-time preview. This will allow artists to see the effects of their changes immediately without having to wait for the offline render to finish. Besides preview, the interface will also contain an interactive timeline to make the process of adding, editing, and blending keyframes fast and smooth. Essentially, we want artists to focus purely on the creative side of the process and not get slowed down by complicated workflows. We're aiming for the release in Q3 2022.
计划
我们的目标是使EbSynth具有交互性和易用性。因此，我们目前专注于速度和用户界面。主要挑战是将性能提高到能够实现实时预览的程度。这将允许美术师立即看到其更改的效果，而无需等待脱机渲染完成。除了预览之外，该界面还将包含一个交互式时间轴，以使添加，编辑和混合关键帧的过程快速流畅。从本质上讲，我们希望艺术家完全专注于流程的创意方面，而不是因复杂的工作流程而减慢速度。我们的目标是在2022年第三季度发布。

The Future of Generative Art
It looks like there is a bit of a bubble around AI today. Anything with the AI or ML label seems sensational, including generative art. So it probably has some future, because people like to discuss it, researchers want to improve it, and artists like to play with it. But I'm not concerned that generative art would actually replace real artists. Aaron Hertzmann wrote a very nice, in-depth essay “Computers do not Make Art, People Do” on this topic.
生成艺术的未来
看起来今天围绕人工智能有一点泡沫。任何带有AI或ML标签的东西似乎都是耸人听闻的，包括生成艺术。因此，它可能有一些未来，因为人们喜欢讨论它，研究人员想要改进它，艺术家喜欢玩它。但我并不担心生成艺术实际上会取代真正的艺术家。Aaron Hertzmann就这个话题写了一篇非常漂亮，深入的文章“计算机不创造艺术，人们会做”。

We noticed style transfer tools based on AI such as Artbreeder or Toonify!
我们注意到基于AI的样式转移工具，例如Artbreeder或Toonify！

Toonify – sometimes it works, sometimes it doesn’t. Take it or leave it.:
Toonify - 有时它有效，有时它不起作用。要就要不要拉倒。:

更多精彩图片 请登录

Currently, the results from tools like these are somewhat unpredictable and hard to control, which limits their utility for production use. However, these tools can be a good source of fun and inspiration, which is positive.
目前，这些工具的结果有些不可预测且难以控制，这限制了它们在生产中的实用性。但是，这些工具可以成为乐趣和灵感的良好来源，这是积极的。