During my PhD, I received a great deal of help and learned lessons worth keeping. I want to share a few of those lessons here in the hope that they may be useful to people beginning their own research work.
01 The Value of Imitation Learning: Make Smarter Decisions
My research field is reinforcement learning. At its core, reinforcement learning designs an agent that improves its decisions through interaction with an environment and eventually reaches an optimal objective. From that perspective, I often feel that life can also be understood as the pursuit of one’s own reward function inside a changing environment.
During primary and secondary education, the environment is relatively fixed, and the reward function is usually simple and externally defined: grades, exams, entrance targets, and so on. But once you enter university or research life, the environment becomes much more complex, and the reward function becomes blurred. At that point, you must actively ask what your own goals are. Do you want to continue studying and explore more possibilities, or do you want to find something you truly love and keep building around it? Those choices deserve real thought.
Unlike reinforcement learning, our lives only happen once. We cannot run massive trial-and-error loops until we discover the optimal policy. But we can still use a form of imitation learning. We can learn from the experiences, reflections, and mistakes of others in order to understand ourselves more efficiently. For people who still feel lost and have not yet figured out their own environment or reward function, my advice is simple: collect information seriously, think carefully, decide firmly, and take responsibility for the outcome.
02 Curiosity, Realism, and Innovation: Explore the Canyon of Science
In research life, keeping curiosity toward the unknown, a realistic commitment to real problems, and a willingness to innovate are what make research both joyful and effective.
Curiosity Is the Starting Point of Real Inquiry
Curiosity is one of the deepest forces behind scientific exploration. I still remember the period when AlphaGo defeated Ke Jie and reinforcement learning triggered a wave of interest across multiplayer electronic games. A representative case was OpenAI Five playing Dota. Because I myself enjoyed playing League of Legends, I naturally became interested in related work and started learning machine learning and PyTorch on my own. That curiosity later became the starting point of my doctoral research. My PhD topic did not literally apply reinforcement learning to Summoner’s Rift, but the interest and early accumulation made it much easier for me to enter the topic and move faster once the real work began.
Realism Is the Foundation of Innovation
Research begins with a deep understanding of existing problems and the relevant field. Only through reading, investigation, and careful data analysis can you identify questions that are truly worth working on. In my doctoral work, reinforcement learning had to interact with distribution-network environments very frequently, which meant repeated power-flow calculations. At the same time, reinforcement learning itself tends to be data-hungry and interaction-heavy. That tension directly led me to the idea of building a distribution-network reinforcement learning environment with much lower computational cost, which eventually became the open-source RL-ADN environment for other researchers to use.
Innovation Is a Series of Expeditions
Scientific innovation feels like exploration. Every new idea has to go through analysis, experimentation, and validation. The road is full of uncertainty and failure, but it is exactly through those failures that we move closer to the truth.
During my PhD, I saw many situations where a reinforcement learning algorithm would fail to converge, or converge in one environment but not another, or stop converging after a single parameter change. I spent many days logging into WandB, reading experiment curves like a detective, and trying to infer what caused the observed behavior. Faced with repeated failures, the only real option was to adjust my mindset and turn repeated defeat into repeated persistence. Over time, that process deepened my understanding of the algorithms and made me much more comfortable with the craft of reinforcement learning. Failure is not the opposite of research. It is one of the normal routes to the boundary of knowledge.
03 Advisors Are a Major Factor in Doctoral Research
The progress and quality of life of a PhD student are strongly correlated with the advisor.
During my doctorate, I did not need to spend much time on work unrelated to research, which meant I had relatively sufficient time for reading, thinking, exploration, and trial and error. That is rare in many engineering environments where projects dominate everything and student time is tightly coupled to advisor incentives.
What matters even more is the training process itself. Frequent meetings in the early stage and targeted research training function like a feedback-regulation system jointly built by the advisor and the doctoral student. The purpose is to train the student into a qualified researcher. This process is not only about building knowledge in a specific domain. More importantly, it is about forming a general research mindset and a durable way of solving problems. That includes how to design research and experiments, how to turn work into publishable papers, and also how to collaborate, negotiate, communicate, present, and make oneself legible to others. That systematic training is extremely important for both research progress and personal growth.
04 Life Beyond Research
As I said earlier, curiosity has long been one of the strongest forces in my life. It was one of the reasons I chose to pursue a doctorate, and it also shapes many of my interests outside research.
I enjoy reading social investigation reports and modern Chinese history, because they reveal the logic of social development and the way major events evolve through competing interests. To me, researchers, investigative journalists, and historians work on different time horizons, but all of them are fundamentally trying to approach the truth. Researchers face the unknown future, journalists try to understand the present, and historians reconstruct the past.
One of the things that attracts me most to research is that it is, at least relatively speaking, less entangled with immediate interest conflicts. That gives it more room for focused inquiry. In research, we can more purely center the problem itself, extract information from fog and large volumes of data, and use logic and analysis to get closer to the future direction of technology and practical solutions. That process has always fascinated me. It satisfies my curiosity, but it also gives me the pleasure of analysis and inference. Whether through research, history, or social inquiry, I keep finding the same reward: a deeper understanding of how the world actually works.